Joint Flashback Adaptation for Forgetting-Resistant Instruction Tuning

2025-05-21

AI Summary

To mitigate catastrophic forgetting in continual instruction tuning of large language models without access to original training data, this paper proposes a novel data-free continual learning framework. Our method introduces lightweight “flashback prompts” to replace conventional data replay, and pioneers a joint flashback adaptation mechanism that couples these prompts with latent-space task interpolation to enable cross-task knowledge transfer. Additionally, we integrate output consistency regularization with adapter-based fine-tuning to preserve task-agnosticity and model compatibility. Evaluated across 1,000+ diverse instruction-following, arithmetic, and general reasoning tasks, our approach achieves substantial improvements in forward transfer to new tasks while reducing backward transfer degradation on old tasks by up to 47%, significantly outperforming state-of-the-art continual learning baselines.

Technology Category

• Machine Learning: Life-Long and Continual Learning
• Search and Optimization: Learning to Search
• Natural Language Processing: (Large) Language Models

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Large language models have achieved remarkable success in various tasks. However, it is challenging for them to learn new tasks incrementally due to catastrophic forgetting. Existing approaches rely on experience replay, optimization constraints, or task differentiation, which encounter strict limitations in real-world scenarios. To address these issues, we propose Joint Flashback Adaptation. We first introduce flashbacks -- a limited number of prompts from old tasks -- when adapting to new tasks and constrain the deviations of the model outputs compared to the original one. We then interpolate latent tasks between flashbacks and new tasks to enable jointly learning relevant latent tasks, new tasks, and flashbacks, alleviating data sparsity in flashbacks and facilitating knowledge sharing for smooth adaptation. Our method requires only a limited number of flashbacks without access to the replay data and is task-agnostic. We conduct extensive experiments on state-of-the-art large language models across 1000+ instruction-following tasks, arithmetic reasoning tasks, and general reasoning tasks. The results demonstrate the superior performance of our method in improving generalization on new tasks and reducing forgetting in old tasks.

Problem

• Prevent catastrophic forgetting in incremental learning of LLMs
• Address limitations of existing replay-based adaptation methods
• Enable joint learning of old and new tasks without replay data

Innovation

• Uses flashbacks to retain old task knowledge
• Interpolates latent tasks for joint learning
• Requires no replay data, task-agnostic approach