Mamba-FSCIL: Dynamic Adaptation with Selective State Space Model for Few-Shot Class-Incremental Learning

2024-07-08

arXiv.org

AI Summary

Few-shot class-incremental learning (FSCIL) confronts a dual challenge: static architectures suffer from overfitting to base classes, while dynamic architectures incur excessive parameter growth. To address this, we propose Dual-SSM—a dual selective state space model framework. It introduces a class-sensitive selective scanning mechanism to decouple feature evolution between base and novel classes, and integrates sequence modeling–driven incremental feature alignment with dynamic weight projection to enable parameter-adaptive expansion and efficient knowledge consolidation. Evaluated on miniImageNet, CUB-200, and CIFAR-100, Dual-SSM consistently surpasses existing state-of-the-art methods. It significantly mitigates catastrophic forgetting and enhances few-shot generalization for novel classes. By jointly achieving parameter efficiency and architectural scalability, Dual-SSM establishes a new paradigm for FSCIL that balances lightweight design with extensibility.

Technology Category

• Machine Learning: Deep Neural Architectures and Foundation Models
• Search and Optimization: Learning to Search
• Computer Vision: Representation Learning for Vision

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization

Abstract

Few-shot class-incremental learning (FSCIL) confronts the challenge of integrating new classes into a model with minimal training samples while preserving the knowledge of previously learned classes. Traditional methods widely adopt static adaptation relying on a fixed parameter space to learn from data that arrive sequentially, prone to overfitting to the current session. Existing dynamic strategies require the expansion of the parameter space continually, leading to increased complexity. In this study, we explore the potential of Selective State Space Models (SSMs) for FSCIL, leveraging its dynamic weights and strong ability in sequence modeling to address these challenges. Concretely, we propose a dual selective SSM projector that dynamically adjusts the projection parameters based on the intermediate features for dynamic adaptation. The dual design enables the model to maintain the robust features of base classes, while adaptively learning distinctive feature shifts for novel classes. Additionally, we develop a class-sensitive selective scan mechanism to guide dynamic adaptation. It minimizes the disruption to base-class representations caused by training on novel data, and meanwhile, forces the selective scan to perform in distinct patterns between base and novel classes. Experiments on miniImageNet, CUB-200, and CIFAR-100 demonstrate that our framework outperforms the existing state-of-the-art methods. The code is available at url{https://github.com/xiaojieli0903/Mamba-FSCIL}.

Problem

• Dynamic adaptation for few-shot class-incremental learning
• Selective State Space Models for knowledge preservation
• Decoupling base and novel class processing dynamically

Innovation

• Selective State Space Models for dynamic adaptation
• Dual selective SSM projector for feature processing
• Class-sensitive selective scan mechanism for adaptation