🤖 AI Summary
This work addresses the challenges of parameter explosion and catastrophic forgetting in class-incremental learning by proposing GRACE, a dynamic scaling framework. GRACE employs a closed-loop “grow–evaluate–compress” mechanism to adaptively regulate model capacity: after expanding the backbone network, it assesses capacity saturation and intelligently decides whether to compress redundant structures, thereby balancing new task acquisition with retention of prior knowledge. The framework innovatively integrates dynamic network expansion, a saturation-aware evaluation algorithm, and model compression techniques to achieve highly efficient memory utilization. Experimental results demonstrate that GRACE achieves state-of-the-art performance across multiple class-incremental benchmarks while reducing memory consumption by up to 73% compared to pure expansion-based approaches.
📝 Abstract
Class Incremental Learning (CIL) poses a fundamental challenge: maintaining a balance between the plasticity required to learn new tasks and the stability needed to prevent catastrophic forgetting. While expansion-based methods effectively mitigate forgetting by adding task-specific parameters, they suffer from uncontrolled architectural growth and memory overhead. In this paper, we propose a novel dynamic scaling framework that adaptively manages model capacity through a cyclic"GRow, Assess, ComprEss"(GRACE) strategy. Crucially, we supplement backbone expansion with a novel saturation assessment phase that evaluates the utilization of the model's capacity. This assessment allows the framework to make informed decisions to either expand the architecture or compress the backbones into a streamlined representation, preventing parameter explosion. Experimental results demonstrate that our approach achieves state-of-the-art performance across multiple CIL benchmarks, while reducing memory footprint by up to a 73% compared to purely expansionist models.