Sparse Deformable Mamba for Hyperspectral Image Classification

2025-04-13

AI Summary

To address inefficiency, detail loss, and computational redundancy in Mamba-based sequence modeling for hyperspectral image classification, this paper proposes the Sparse Deformable Mamba (SD-Mamba) framework. Methodologically, SD-Mamba introduces: (1) a novel sparse deformable sequence construction mechanism that adaptively subsamples spectral-spatial tokens to drastically reduce sequence length; (2) a dual-path deformable Mamba module jointly modeling spatial and spectral dimensions in a decoupled manner; and (3) an attention-guided feature fusion strategy to enhance discriminative representation of salient regions. Evaluated on multiple benchmark hyperspectral datasets, SD-Mamba achieves new state-of-the-art performance with significantly fewer parameters and lower latency: it improves mean overall accuracy by 1.2–2.8%, boosts few-shot class recognition accuracy by 4.5%, and accelerates inference speed by 37% compared to existing methods.

Technology Category

• Machine Learning: Learning with Manifolds
• Computer Vision: Multi-modal Vision
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

Although the recent Mamba models significantly improve hyperspectral image (HSI) classification, one critical challenge is caused by the difficulty to build the Mamba sequence efficiently and effectively. This paper presents a Sparse Deformable Mamba (SDMamba) approach for enhanced HSI classification, with the following contributions. First, to enhance Mamba sequence, an efficient Sparse Deformable Sequencing (SDS) approach is designed to adaptively learn the"optimal"sequence, leading to sparse and deformable Mamba sequence with increased detail preservation and decreased computations. Second, to boost spatial-spectral feature learning, based on SDS, a Sparse Deformable Spatial Mamba Module (SDSpaM) and a Sparse Deformable Spectral Mamba Module (SDSpeM) are designed for tailored modeling of the spatial information spectral information. Last, to improve the fusion of SDSpaM and SDSpeM, an attention based feature fusion approach is designed to integrate the outputs of the SDSpaM and SDSpeM. The proposed method is tested on several benchmark datasets with many state-of-the-art approaches, demonstrating that the proposed approach can achieve higher accuracy, faster speed, and better detail small-class preservation capability.

Problem

• Efficiently building Mamba sequences for HSI classification
• Enhancing spatial-spectral feature learning in HSI
• Improving fusion of spatial and spectral Mamba modules

Innovation

• Sparse Deformable Sequencing for optimal Mamba sequence
• Spatial-Spectral Mamba Modules for tailored feature learning
• Attention-based fusion for improved feature integration