STAR: Stability-Inducing Weight Perturbation for Continual Learning

2025-03-03

AI Summary

In continual learning, models suffer from catastrophic forgetting when acquiring new tasks. To address this, we propose the STAR loss function, the first to incorporate robust optimization principles into continual learning loss design. STAR enforces prediction consistency within a local parameter neighborhood via KL-divergence-sensitive adversarial weight perturbations, thereby enhancing model stability and knowledge retention. Unlike memory-based approaches, STAR requires no additional sample storage and can be seamlessly integrated—end-to-end—with mainstream rehearsal methods (e.g., ER, DER). Extensive experiments across multiple benchmarks demonstrate that STAR consistently improves rehearsal-based baselines by an average of 15% in accuracy, significantly reduces forgetting rates, and achieves performance on par with or surpassing state-of-the-art rehearsal-augmented methods.

Technology Category

• Machine Learning: Adversarial Learning & Robustness
• Search and Optimization: Learning to Search
• Intelligent Robots: Learning & Optimization for ROB

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Humans can naturally learn new and varying tasks in a sequential manner. Continual learning is a class of learning algorithms that updates its learned model as it sees new data (on potentially new tasks) in a sequence. A key challenge in continual learning is that as the model is updated to learn new tasks, it becomes susceptible to catastrophic forgetting, where knowledge of previously learned tasks is lost. A popular approach to mitigate forgetting during continual learning is to maintain a small buffer of previously-seen samples and to replay them during training. However, this approach is limited by the small buffer size, and while forgetting is reduced, it is still present. In this paper, we propose a novel loss function, STAR, that exploits the worst-case parameter perturbation that reduces the KL-divergence of model predictions with that of its local parameter neighborhood to promote stability and alleviate forgetting. STAR can be combined with almost any existing rehearsal-based method as a plug-and-play component. We empirically show that STAR consistently improves the performance of existing methods by up to 15% across varying baselines and achieves superior or competitive accuracy to that of state-of-the-art methods aimed at improving rehearsal-based continual learning.

Problem

• Addresses catastrophic forgetting in continual learning.
• Proposes STAR loss to stabilize model predictions.
• Enhances rehearsal-based methods to reduce forgetting.

Innovation

• STAR: novel loss function for stability
• Reduces KL-divergence via parameter perturbation
• Plug-and-play with rehearsal-based methods