MSSR: Memory-Aware Adaptive Replay for Continual LLM Fine-Tuning

2026-03-10

AI Summary

This work addresses the challenge of catastrophic forgetting in large language models during continual fine-tuning, a problem inadequately mitigated by existing replay methods that either rely on heuristic rules or incur high computational costs. Inspired by human memory mechanisms, the authors propose the Memory Strength–aware Sample Replay (MSSR) framework, which dynamically optimizes both the content and timing of replayed samples through sample-level memory strength estimation and adaptive replay scheduling. By integrating experience replay, memory strength modeling, and a lightweight scheduling algorithm, MSSR effectively balances the mitigation of forgetting with rapid adaptation to new tasks. Extensive experiments across three backbone models and eleven sequential tasks demonstrate that MSSR significantly outperforms current replay strategies, with particularly notable gains on reasoning-intensive and multiple-choice tasks.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Planning, Routing, and Scheduling: Planning with Language Models
• Multiagent Systems: Multiagent Learning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Continual fine-tuning of large language models (LLMs) is becoming increasingly crucial as these models are deployed in dynamic environments where tasks and data distributions evolve over time. While strong adaptability enables rapid acquisition of new knowledge, it also exposes LLMs to catastrophic forgetting, where previously learned skills degrade during sequential training. Existing replay-based strategies, such as fixed interleaved replay, accuracy-supervised, and loss-driven scheduling, remain limited: some depend on heuristic rules and provide only partial mitigation of forgetting, while others improve performance but incur substantial computational overhead. Motivated by retention dynamics under sequential fine-tuning, we propose Memory-Inspired Sampler and Scheduler Replay (MSSR), an experience replay framework that estimates sample-level memory strength and schedules rehearsal at adaptive intervals to mitigate catastrophic forgetting while maintaining fast adaptation. Extensive experiments across three backbone models and 11 sequential tasks show that MSSR consistently outperforms state-of-the-art replay baselines, with particularly strong gains on reasoning-intensive and multiple-choice benchmarks.

Problem

• continual fine-tuning
• catastrophic forgetting
• large language models
• experience replay
• memory retention

Innovation

• continual learning
• catastrophic forgetting
• experience replay
• memory-aware scheduling
• adaptive rehearsal