From Isolation to Integration: Building an Adaptive Expert Forest for Pre-Trained Model-based Class-Incremental Learning

2026-02-24

AI Summary

This work addresses catastrophic forgetting and performance bottlenecks in pretrained model–based class-incremental learning, which stem from isolated task-specific knowledge and insufficient semantic relationships across tasks. To this end, we propose Semantic-guided Adaptive Expert Forest (SAEF), a novel approach that clusters task adapters according to semantic similarity and organizes them into a balanced expert tree. SAEF dynamically activates relevant experts and fuses their outputs through confidence-weighted aggregation, enabling efficient knowledge sharing and transfer. Combined with a frozen pretrained backbone, SAEF achieves state-of-the-art performance across multiple class-incremental benchmarks, significantly outperforming existing methods and establishing a paradigm shift from isolated learning to semantically coordinated adaptation.

Technology Category

• Machine Learning: Mixture of Experts (MoE)
• Cognitive Modeling & Cognitive Systems: Adaptive Behavior
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Class-Incremental Learning (CIL) requires models to learn new classes without forgetting old ones. A common method is to freeze a pre-trained model and train a new, lightweight adapter for each task. While this prevents forgetting, it treats the learned knowledge as a simple, unstructured collection and fails to use the relationships between tasks. To this end, we propose the Semantic-guided Adaptive Expert Forest (SAEF), a new method that organizes adapters into a structured hierarchy for better knowledge sharing. SAEF first groups tasks into conceptual clusters based on their semantic relationships. Then, within each cluster, it builds a balanced expert tree by creating new adapters from merging the adapters of similar tasks. At inference time, SAEF finds and activates a set of relevant experts from the forest for any given input. The final prediction is made by combining the outputs of these activated experts, weighted by how confident each expert is. Experiments on several benchmark datasets show that SAEF achieves SOTA performance.

Problem

• Class-Incremental Learning
• Pre-Trained Models
• Knowledge Sharing
• Task Relationships
• Adapter Isolation

Innovation

• Class-Incremental Learning
• Expert Forest
• Semantic Clustering
• Adapter Merging
• Pre-trained Models