Do Your Best and Get Enough Rest for Continual Learning

2025-03-24

AI Summary

Continual learning faces the core challenge of retaining newly acquired knowledge over extended periods while mitigating catastrophic forgetting. Inspired by the Ebbinghaus forgetting curve, this paper proposes the “view-batch” model—the first to formalize the human memory consolidation mechanism of rest intervals as an optimizable, dynamically scheduled recall period. We design an interval-aware experience replay strategy and a single-sample multi-view self-supervised learning paradigm, jointly optimizing both the timing of sample retraining and representation learning. Under diverse continual learning protocols—including task-, class-, and domain-incremental settings—our approach consistently outperforms state-of-the-art methods. Empirical results demonstrate significant improvements in long-term memory retention and robustness across extended task sequences. The implementation is publicly available.

Technology Category

• Machine Learning: Life-Long and Continual Learning
• Computer Vision: Learning & Optimization for CV
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

According to the forgetting curve theory, we can enhance memory retention by learning extensive data and taking adequate rest. This means that in order to effectively retain new knowledge, it is essential to learn it thoroughly and ensure sufficient rest so that our brain can memorize without forgetting. The main takeaway from this theory is that learning extensive data at once necessitates sufficient rest before learning the same data again. This aspect of human long-term memory retention can be effectively utilized to address the continual learning of neural networks. Retaining new knowledge for a long period of time without catastrophic forgetting is the critical problem of continual learning. Therefore, based on Ebbinghaus' theory, we introduce the view-batch model that adjusts the learning schedules to optimize the recall interval between retraining the same samples. The proposed view-batch model allows the network to get enough rest to learn extensive knowledge from the same samples with a recall interval of sufficient length. To this end, we specifically present two approaches: 1) a replay method that guarantees the optimal recall interval, and 2) a self-supervised learning that acquires extensive knowledge from a single training sample at a time. We empirically show that these approaches of our method are aligned with the forgetting curve theory, which can enhance long-term memory. In our experiments, we also demonstrate that our method significantly improves many state-of-the-art continual learning methods in various protocols and scenarios. We open-source this project at https://github.com/hankyul2/ViewBatchModel.

Problem

• Address catastrophic forgetting in continual learning networks
• Optimize recall intervals for better memory retention
• Enhance long-term learning via rest-adjusted training schedules

Innovation

• View-batch model optimizes recall intervals
• Replay method ensures optimal recall intervals
• Self-supervised learning from single samples