When Does Continual Learning Require Learning

2026-07-08

AI Summary

This study addresses the dual challenges of emerging domains and data drift faced by large language models in dynamic environments. The authors decompose continual learning into spatial (new domains) and temporal (data drift) dimensions and propose the first mechanism-agnostic, unified evaluation protocol. Within this consistent framework, they systematically compare the adaptability of diverse approaches—including prompt engineering (e.g., GEPA, ACE), supervised fine-tuning (SFT, SDFT), online reinforcement learning (GRPO, SDPO), and context compression (Cartridges, In-place TTT). Their analysis reveals that effective adaptation depends on aligning update mechanisms with specific environmental dynamics: online reinforcement learning excels at knowledge updating yet is sensitive to noise; distillation-based methods offer stability but struggle to correct outdated facts; and prompt-based strategies respond rapidly but exhibit limited generalization.

Abstract

As large language models (LLMs) become increasingly capable, the next question is how can we enable models to continually learn? Today, the field largely frames this as a problem of context management and mitigating forgetting. We argue this framing is incomplete: continual learning is fundamentally about increasing model competence as the world changes. We disentangle this change along two axes -- space, where the model encounters new domains, and time, where the underlying data drifts under a fixed task. This framing lets us study continual learning under realistic conditions: new domains arrive over time, facts drift past their training cutoff, and agentic interactions accumulate state across episodes. To evaluate methods under this setting, we recast widely used LLM benchmarks as sequential problems and introduce a single mechanism-agnostic protocol that compares prompt-based methods (GEPA, ACE), supervised learning (SFT, SDFT), reinforcement learning (GRPO, SDPO), and context compression (Cartridges, In-place TTT). Prompt-based methods fit each new stage quickly but degrade on future tasks. Distillation-based methods accumulate knowledge stably but struggle to update outdated facts. Context compression improves efficiency without substantially improving the ability to learn new tasks. Online reinforcement learning adapts most effectively to knowledge updates but remains sensitive to noisy reward signals. Overall, our results suggest that continual learning is not a single capability: different patterns of environmental change require fundamentally different update behaviors, determining when adaptation must be learned inside model weights and when it can be achieved through external scaffolding. We hope that understanding where each method succeeds and fails will guide the design of stronger continual learning systems.

Problem

• continual learning
• large language models
• data drift
• domain shift
• knowledge updating

Innovation

• continual learning
• large language models
• environmental change
• knowledge updating
• evaluation protocol