Few-Shot Object Detection via Spatial-Channel State Space Model

2025-07-21

AI Summary

In few-shot object detection (FSOD), scarce training samples lead to inaccurate channel-wise feature responses—high-weight channels may be ineffective, while low-weight ones may contain critical discriminative information. To address this, we propose a spatial-channel collaborative state space modeling framework. We introduce the Mamba architecture into FSOD for the first time, designing a Channel State Modeling (CSM) module that treats channels as sequential tokens to capture long-range inter-channel dependencies. Complementing this, a Spatial Feature Modeling (SFM) module jointly optimizes feature responses across both spatial and channel dimensions. Our method dynamically enhances informative channels and suppresses spurious activations, thereby improving feature discriminability and generalization. Evaluated on PASCAL VOC and MS COCO, our approach achieves state-of-the-art performance, particularly excelling in ultra-low-shot regimes (1-shot and 2-shot), where it demonstrates markedly improved feature focusing capability.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Search and Optimization: Learning to Search

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization

Abstract

Due to the limited training samples in few-shot object detection (FSOD), we observe that current methods may struggle to accurately extract effective features from each channel. Specifically, this issue manifests in two aspects: i) channels with high weights may not necessarily be effective, and ii) channels with low weights may still hold significant value. To handle this problem, we consider utilizing the inter-channel correlation to facilitate the novel model's adaptation process to novel conditions, ensuring the model can correctly highlight effective channels and rectify those incorrect ones. Since the channel sequence is also 1-dimensional, its similarity with the temporal sequence inspires us to take Mamba for modeling the correlation in the channel sequence. Based on this concept, we propose a Spatial-Channel State Space Modeling (SCSM) module for spatial-channel state modeling, which highlights the effective patterns and rectifies those ineffective ones in feature channels. In SCSM, we design the Spatial Feature Modeling (SFM) module to balance the learning of spatial relationships and channel relationships, and then introduce the Channel State Modeling (CSM) module based on Mamba to learn correlation in channels. Extensive experiments on the VOC and COCO datasets show that the SCSM module enables the novel detector to improve the quality of focused feature representation in channels and achieve state-of-the-art performance.

Problem

• Improving feature extraction in few-shot object detection
• Balancing channel weights to highlight effective features
• Modeling inter-channel correlations using Spatial-Channel State Space

Innovation

• Utilizes inter-channel correlation for feature adaptation
• Employs Mamba for channel sequence correlation modeling
• Balances spatial and channel learning via SFM and CSM