CI-CBM: Class-Incremental Concept Bottleneck Model for Interpretable Continual Learning

2026-04-15

AI Summary

This work addresses the challenge of balancing catastrophic forgetting and model interpretability in class-incremental continual learning by proposing an interpretable continual learning framework based on concept bottlenecks. By integrating concept regularization and a pseudo-concept generation mechanism, the method effectively preserves human-understandable concept representations throughout the incremental learning process, thereby maintaining the integrity of the concept bottleneck architecture under class-incremental settings for the first time. The approach supports both pre-trained and from-scratch training paradigms, achieving an average accuracy improvement of 36% over existing interpretable methods across seven benchmark datasets—matching the performance of black-box models while providing both input-level and global-level interpretable decision rules.

Technology Category

• Machine Learning: Life-Long and Continual Learning
• Computer Vision: Interpretability, Explainability, and Transparency
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Catastrophic forgetting remains a fundamental challenge in continual learning, in which models often forget previous knowledge when fine-tuned on a new task. This issue is especially pronounced in class incremental learning (CIL), which is the most challenging setting in continual learning. Existing methods to address catastrophic forgetting often sacrifice either model interpretability or accuracy. To address this challenge, we introduce ClassIncremental Concept Bottleneck Model (CI-CBM), which leverage effective techniques, including concept regularization and pseudo-concept generation to maintain interpretable decision processes throughout incremental learning phases. Through extensive evaluation on seven datasets, CI-CBM achieves comparable performance to black-box models and outperforms previous interpretable approaches in CIL, with an average 36% accuracy gain. CICBM provides interpretable decisions on individual inputs and understandable global decision rules, as shown in our experiments, thereby demonstrating that human understandable concepts can be maintained during incremental learning without compromising model performance. Our approach is effective in both pretrained and non-pretrained scenarios; in the latter, the backbone is trained from scratch during the first learning phase. Code is publicly available at github.com/importAmir/CI-CBM.

Problem

• catastrophic forgetting
• class-incremental learning
• continual learning
• model interpretability
• accuracy

Innovation

• Class-Incremental Learning
• Concept Bottleneck Model
• Interpretable AI
• Catastrophic Forgetting
• Pseudo-Concept Generation