GeRe: Towards Efficient Anti-Forgetting in Continual Learning of LLM via General Samples Replay

2025-08-06

AI Summary

Large language models (LLMs) suffer catastrophic forgetting during cross-domain continual fine-tuning—degrading both general-purpose capabilities and performance on previously learned tasks. To address this dual forgetting, we propose GeRe, a novel framework that leverages only a small set of pre-collected generic text samples for replay. Its core innovation is a threshold-based activation state constraint mechanism, coupled with a threshold-based margin (TM) loss designed to enhance neural state consistency across tasks. GeRe significantly outperforms baseline strategies—including KL divergence, L1/L2 regularization, and logit distillation—in mitigating both types of forgetting. Extensive experiments demonstrate that GeRe preserves model generalization while consistently improving continual task accuracy and robustness. Notably, this work provides the first empirical validation that generic-sample replay can simultaneously suppress both general-capability degradation and historical-task forgetting—a key advance in continual learning for LLMs.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

The continual learning capability of large language models (LLMs) is crucial for advancing artificial general intelligence. However, continual fine-tuning LLMs across various domains often suffers from catastrophic forgetting, characterized by: 1) significant forgetting of their general capabilities, and 2) sharp performance declines in previously learned tasks. To simultaneously address both issues in a simple yet stable manner, we propose General Sample Replay (GeRe), a framework that use usual pretraining texts for efficient anti-forgetting. Beyond revisiting the most prevalent replay-based practices under GeRe, we further leverage neural states to introduce a enhanced activation states constrained optimization method using threshold-based margin (TM) loss, which maintains activation state consistency during replay learning. We are the first to validate that a small, fixed set of pre-collected general replay samples is sufficient to resolve both concerns--retaining general capabilities while promoting overall performance across sequential tasks. Indeed, the former can inherently facilitate the latter. Through controlled experiments, we systematically compare TM with different replay strategies under the GeRe framework, including vanilla label fitting, logit imitation via KL divergence and feature imitation via L1/L2 losses. Results demonstrate that TM consistently improves performance and exhibits better robustness. Our work paves the way for efficient replay of LLMs for the future. Our code and data are available at https://github.com/Qznan/GeRe.

Problem

• Address catastrophic forgetting in continual LLM learning
• Retain general capabilities during sequential task training
• Enhance performance stability via sample replay optimization

Innovation

• General Sample Replay (GeRe) for anti-forgetting
• Threshold-based margin (TM) loss optimization
• Fixed small set of general replay samples