Continual Extraction of Semantic Relations using Augmented Prototypes with Energy-based Model Alignment

Abstract

Continual relation extraction (CRE) is a critical task in natural language processing that
aims to learn new relation types incrementally while preserving knowledge of previously learned
relations. However, existing CRE models often struggle with catastrophic forgetting and inefficient
utilization of memory. In this paper, we propose a CRE model that leverages class-specific prototypes
and energy-based latent alignment to address these challenges. Our approach stores relation prototypes instead of real data points, enriching them with Gaussian noise during training. We incorporate
contrastive learning to obtain effective representations for memory prototype data and introduce an
Energy-based Latent feature space Alignment (ELI) module to mitigate representational shift across
tasks. We evaluate our model on two benchmark datasets: FewRel, a balanced few-shot relation
classification dataset, and TACRED, a large-scale imbalanced relation extraction dataset. Extensive
experiments demonstrate that our proposed method consistently outperforms state-of-the-art CRE
models across multiple tasks, with improvements of up to 4% over existing methods. This consistent
superior performance highlights our model effectiveness in addressing the challenges of continual relation extraction, particularly in maintaining performance across a sequence of tasks while mitigating
catastrophic forgetting.