Non-Forgetting Knowledge Allocation with Bi-level Competition for Class-Incremental Learning

2026-05-28

AI Summary

This work addresses the limitations of existing knowledge allocation mechanisms in class-incremental learning, where pretrained models often neglect task-specific differences and suffer from allocator forgetting. To overcome these issues, the authors propose a forgetting-resistant allocator incorporating a two-level competition scheme: an intra-task winner-take-all strategy dynamically activates task-adapted adapters, while an inter-task last-place elimination mechanism suppresses interference across tasks. The allocator is efficiently trained via recursive least squares and further stabilized through a dedicated stability-enhancement procedure. Empirical results demonstrate that the proposed method significantly improves performance stability on previously learned tasks, achieving allocation effectiveness comparable to that of full-data training—even when using only incrementally available data—thereby enhancing both personalization and anti-forgetting efficiency in adapter-based knowledge utilization.

Abstract

Class-Incremental Learning (CIL) with pre-trained models (PTMs) aims to sequentially adapt PTMs to new categories without forgetting old knowledge. Built upon PTMs, existing adapter-based methods mainly train models via distinct task-specific adapters, and present a uniform knowledge allocation for each adapter during inference. However, this allocation mechanism ignores the nature of task discrepancy and leads to suboptimal utilization of adapters. Also, under CIL constraint, an allocator is prone to forgetting when tasks evolve. To address these issues, we propose a Non-Forgetting Allocation with Bi-Level Competition (NoFA-BC). NoFA-BC constructs a non-forgetting allocator (NFA) by transforming the allocator training into a recursive least-squares problem and achieves an allocator equivalent to that trained with all data. Based on the NFA, a Bi-Level Competition (BLC) including an intra-task level Winner-Takes-All (WTA) mechanism and inter-task Last-Ones-Fall (LOF) elimination is proposed to provide better allocation of adapter knowledge. WTA extracts the most significant logit within a task to represent the adapter's contribution and LOF suppresses the irrelevant adapters. With BLC, participation ratio of each adapter can be tailored for each input. Moreover, a Stability Enhancement (SE) process is incorporated to further improve the performance of old tasks.

Problem

• Class-Incremental Learning
• Knowledge Allocation
• Adapter-based Methods
• Catastrophic Forgetting
• Task Discrepancy

Innovation

• Class-Incremental Learning
• Adapter-based Methods
• Non-Forgetting Allocator
• Bi-Level Competition
• Recursive Least-Squares