Forgetting Order of Continual Learning: Examples That are Learned First are Forgotten Last

2024-06-14

arXiv.org

AI Summary

Catastrophic forgetting remains a fundamental challenge in continual learning. This work investigates sample-level forgetting sensitivity and identifies a strong correlation between learning order and forgetting severity: samples learned earlier exhibit greater resistance to forgetting. Motivated by this finding, we propose the “Goldilocks” sampling principle—selecting only moderately learned samples for rehearsal while excluding those learned too quickly or too slowly. We further design a training-dynamics-based model to estimate per-sample learning speed and integrate it into a dynamic buffer update mechanism. Our approach is lightweight and seamlessly compatible with mainstream rehearsal methods (e.g., ER, A-GEM). Extensive experiments on Split-CIFAR10/100 and Split-ImageNet demonstrate state-of-the-art performance: average accuracy improves by 2.1–3.7%, and forgetting rates decrease significantly. To our knowledge, this is the first work to quantitatively establish the relationship between learning timing and forgetting, introducing a novel sample-aware paradigm for continual learning.

Technology Category

• Machine Learning: Life-Long and Continual Learning
• Search and Optimization: Learning to Search
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Catastrophic forgetting poses a significant challenge in continual learning, where models often forget previous tasks when trained on new data. Our empirical analysis reveals a strong correlation between catastrophic forgetting and the learning speed of examples: examples learned early are rarely forgotten, while those learned later are more susceptible to forgetting. We demonstrate that replay-based continual learning methods can leverage this phenomenon by focusing on mid-learned examples for rehearsal. We introduce Goldilocks, a novel replay buffer sampling method that filters out examples learned too quickly or too slowly, keeping those learned at an intermediate speed. Goldilocks improves existing continual learning algorithms, leading to state-of-the-art performance across several image classification tasks.

Problem

• Predicting susceptibility to catastrophic forgetting in neural networks
• Investigating replay buffer composition impact on forgetting
• Proposing Speed-Based Sampling to improve continual learning performance

Innovation

• Uses learning speed to predict forgetting
• Introduces Speed-Based Sampling (SBS) strategy
• Improves replay buffer composition for performance