Feature Calibration enhanced Parameter Synthesis for CLIP-based Class-incremental Learning

2025-03-24

AI Summary

This work addresses the dual challenges of catastrophic forgetting and degraded generalization in class-incremental learning (CIL) with CLIP models. We propose a feature-calibration-enhanced incremental parameter synthesis method that adaptively modulates the contribution weight of original visual features to classification while dynamically fusing learnable parameters across tasks—jointly enabling effective acquisition of novel-class knowledge and robust retention of old-class knowledge. Unlike existing approaches, our method requires neither raw data storage nor auxiliary prompt engineering; instead, it performs lightweight, task-aware adaptation solely upon CLIP’s pre-trained vision-language joint representations. Evaluated on standard CIL benchmarks—including CIFAR-100 and ImageNet-100—our approach achieves significant improvements over state-of-the-art methods: +3.2% in average accuracy and −4.7% in forgetting rate. Moreover, it demonstrates strong cross-domain generalization capability, confirming its effectiveness and scalability in practical continual learning scenarios.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Computer Vision: Diffusion Models for Vision
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Class-incremental Learning (CIL) enables models to continuously learn new class knowledge while memorizing previous classes, facilitating their adaptation and evolution in dynamic environments. Traditional CIL methods are mainly based on visual features, which limits their ability to handle complex scenarios. In contrast, Vision-Language Models (VLMs) show promising potential to promote CIL by integrating pretrained knowledge with textual features. However, previous methods make it difficult to overcome catastrophic forgetting while preserving the generalization capabilities of VLMs. To tackle these challenges, we propose Feature Calibration enhanced Parameter Synthesis (FCPS) in this paper. Specifically, our FCPS employs a specific parameter adjustment mechanism to iteratively refine the proportion of original visual features participating in the final class determination, ensuring the model's foundational generalization capabilities. Meanwhile, parameter integration across different tasks achieves a balance between learning new class knowledge and retaining old knowledge. Experimental results on popular benchmarks (e.g., CIFAR100 and ImageNet100) validate the superiority of the proposed method.

Problem

• Enhancing class-incremental learning with feature calibration
• Overcoming catastrophic forgetting in vision-language models
• Balancing new knowledge acquisition with old knowledge retention

Innovation

• Feature Calibration enhances parameter synthesis for CIL
• Parameter adjustment refines visual feature proportions
• Balances new class learning with old knowledge retention