To Retain or to Adapt? Generalizing Continual Learning

2026-07-06

AI Summary

This work addresses the limitation of traditional continual learning, which overly emphasizes preserving old knowledge to approximate joint training while neglecting real-time adaptation to new tasks in non-stationary environments. The problem is reformulated as an online optimization framework, with average lifelong error as the performance metric, and a notion of transfer efficiency is introduced to characterize the trade-off between stability and transient error induced by historical knowledge. Drawing on critical task duration theory, the study identifies conditions under which past knowledge shifts from beneficial to detrimental. Building on this insight, the paper proposes a predictive continual learning paradigm that dynamically models future tasks to optimize expected performance. Integrating online learning, transfer efficiency analysis, and convergence theory, an algorithm based on task-window interpolation is developed and validated on image classification and reinforcement learning benchmarks, demonstrating significant superiority over both joint training and independent learning, especially under distribution shift.

Abstract

The Continual Learning (CL) literature has long been driven by the goal of mitigating catastrophic forgetting. This objective rests on a pervasive, often unstated assumption: that a lifelong learner should approximate the Joint-Task Learning (JTL) solution and retain all previously acquired knowledge. We challenge this retention-centered premise, arguing that in non-stationary environments prioritizing retention can impede real-time adaptation. Shifting the focus to the Average Lifelong Error (ALE), we formalize CL as an online optimization problem governed by the interaction between environmental and learning dynamics. We introduce Transfer Efficiency as a quantitative measure of the tension between Instability, the bias inherited from conflicting past experience, and Transient Error, the optimization cost of learning new tasks from scratch. Under mild convergence conditions, holding across linear and neural network models, this decomposition yields a Critical Task Duration: a closed-form threshold beyond which historical knowledge transitions from a warm-start advantage to an optimization liability whenever retention induces a positive stationary bias. We validate these theoretical predictions on continual image classification and reinforcement learning benchmarks. Finally, by connecting continual learning to the online learning framework of predictable sequences, we show that JTL is only one instance of a broader family of objectives, and we propose a new general class of continual learning algorithms, which we call Predictive Continual Learning. Predictive CL algorithms optimize expected future performance under an explicit, dynamically updated model of future tasks. As a proof of concept, we analyze a Window algorithm that interpolates between JTL and Independent-Task Learning (ITL), outperforming both under controlled distributional drift.

Problem

• Continual Learning
• Catastrophic Forgetting
• Non-stationary Environments
• Joint-Task Learning
• Adaptation

Innovation

• Predictive Continual Learning
• Average Lifelong Error
• Transfer Efficiency
• Critical Task Duration
• Non-stationary Environments