Cost-Efficient Continual Learning with Sufficient Exemplar Memory

2025-02-11

AI Summary

This work addresses the computational efficiency optimization problem in continual learning (CL) under the realistic large-model-era setting where GPU compute is expensive but memory is abundant—contrary to the traditional CL assumption of extremely limited memory budgets. We propose a novel weight-space-oriented paradigm comprising selective weight resetting, cross-task weight averaging, and efficient exemplar replay, enabling joint acceleration of training and improvement of generalization under relaxed memory constraints. Extensive experiments on mainstream CL benchmarks demonstrate that our method achieves state-of-the-art performance while reducing GPU compute cost to only 25–33% of existing approaches. The proposed framework establishes a scalable, cost-effective baseline for practical continual learning, bridging the gap between theoretical CL assumptions and modern hardware realities.

Technology Category

• Machine Learning: Life-Long and Continual Learning
• Search and Optimization: Learning to Search
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

Continual learning (CL) research typically assumes highly constrained exemplar memory resources. However, in many real-world scenarios-especially in the era of large foundation models-memory is abundant, while GPU computational costs are the primary bottleneck. In this work, we investigate CL in a novel setting where exemplar memory is ample (i.e., sufficient exemplar memory). Unlike prior methods designed for strict exemplar memory constraints, we propose a simple yet effective approach that directly operates in the model's weight space through a combination of weight resetting and averaging techniques. Our method achieves state-of-the-art performance while reducing the computational cost to a quarter or third of existing methods. These findings challenge conventional CL assumptions and provide a practical baseline for computationally efficient CL applications.

Problem

• Optimize continual learning computational costs
• Utilize ample exemplar memory efficiently
• Challenge conventional memory constraints assumptions

Innovation

• Weight resetting techniques
• Weight averaging methods
• Reduced computational cost