GR4CIL: Gap-compensated Routing for CLIP-based Class Incremental Learning

2026-04-20

AI Summary

This work addresses the challenges of catastrophic forgetting and poor cross-task response calibration in class-incremental learning with CLIP, which arise from its shared-parameter architecture. To mitigate these issues, the authors propose a task-aware routing mechanism that jointly preserves task-specific visual knowledge while stabilizing the shared textual semantic space. By incorporating an orthogonal compensation strategy, the method effectively reduces modality gap bias, enhances intra-task discriminability, and widens inter-task confidence margins. Extensive experiments demonstrate that the proposed approach significantly outperforms strong existing baselines across multiple benchmarks, all while retaining CLIP’s zero-shot generalization capability.

Technology Category

• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Class-Incremental Learning (CIL) aims to continuously acquire new categories while preserving previously learned knowledge. Recently, Contrastive Language-Image Pre-trained (CLIP) models have shown strong potential for CIL due to their powerful generalization ability. However, existing methods still face two key challenges: shared-parameter adaptation tends to cause old-knowledge drift, and task-specific knowledge organization often leads to poorly calibrated cross-task responses, making reliable routing difficult. To address these issues, we propose GR4CIL, a framework combining task discrimination and knowledge routing for CLIP-based CIL. GR4CIL preserves task-specific visual knowledge while maintaining an incrementally stable shared textual semantic space, thereby reducing interference across tasks. Moreover, we introduce an orthogonal compensation mechanism to mitigate modality-gap-induced bias, enhance within-task discrimination, and enlarge the score margin between the ground-truth task and competing tasks. As a result, GR4CIL enables more reliable task-aware routing over learned knowledge while retaining the zero-shot generalization capability. Experiments on multiple benchmarks show that GR4CIL consistently outperforms strong baselines.

Problem

• Class-Incremental Learning
• CLIP
• knowledge drift
• task routing
• modality gap

Innovation

• Class-Incremental Learning
• CLIP
• Knowledge Routing
• Modality Gap Compensation
• Task-aware Discrimination