Continual Extraction of Semantic Relations using Augmented Prototypes with Energy-based Model Alignment - Dang, Thanh Hai; -Trang Pham Thi, Quynh; -Hung Nguyen, Duc; -Thanh Nguyen, Tri; -Trong Le, Duc (VNU Journal of Science: Computer Science and Communication Engineering; Vol 40 No 2 (2024), 2024)
Continual relation extraction (CRE) is a critical task in natural language processing that aims to learn new relation types incrementally while preserving knowledge of previously learned relations. However, existing CRE models often struggle with catastrophic forgetting and ine ffi cient utilization of memory. In this paper, we propose a CRE model that leverages class-specific prototypes and energy-based latent alignment to address these challenges. Our approach stores relation prototypes instead of real data points, enriching them with Gaussian noise during training. We incorporate contrastive learning to obtain e ff ective representations for memory prototype data and introduce an Energy-based Latent feature space Alignment (ELI) module to mitigate representational shift across tasks. We evaluate our model on two benchmark datasets: FewRel, a balanced few-shot relation classification dataset, and TACRED, a large-scale imbalanced relation extraction dataset. Extensive experiments demonstrate that our proposed method consistently outperforms state-of-the-art CRE models across multiple tasks, with improvements of up to 4% over existing methods. This consistent superior performance highlights our model e ff ectiveness in addressing the challenges of continual relation extraction, particularly in maintaining performance across a sequence of tasks while mitigating catastrophic forgetting.