M3SR: Multi-Scale Multi-Perceptual Mamba for Efficient Spectral Reconstruction

2026-01-13

AI Summary

Existing spectral reconstruction methods are limited by their reliance on single-scale spatial perception and feature extraction, hindering their ability to adequately model the complex structures inherent in hyperspectral images. To address this challenge, this work proposes the M3SR architecture, which for the first time integrates multi-scale and multi-perception mechanisms into a Mamba-based framework. Specifically, we design a Mamba-based multi-perception fusion module and embed it within a U-Net structure to enable efficient extraction and fusion of global, intermediate, and local multi-scale features. Extensive experiments demonstrate that the proposed method significantly outperforms current state-of-the-art approaches across multiple benchmark datasets, achieving higher reconstruction accuracy while substantially reducing computational overhead.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multi-instance/Multi-view Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

The Mamba architecture has been widely applied to various low-level vision tasks due to its exceptional adaptability and strong performance. Although the Mamba architecture has been adopted for spectral reconstruction, it still faces the following two challenges: (1) Single spatial perception limits the ability to fully understand and analyze hyperspectral images; (2) Single-scale feature extraction struggles to capture the complex structures and fine details present in hyperspectral images. To address these issues, we propose a multi-scale, multi-perceptual Mamba architecture for the spectral reconstruction task, called M3SR. Specifically, we design a multi-perceptual fusion block to enhance the ability of the model to comprehensively understand and analyze the input features. By integrating the multi-perceptual fusion block into a U-Net structure, M3SR can effectively extract and fuse global, intermediate, and local features, thereby enabling accurate reconstruction of hyperspectral images at multiple scales. Extensive quantitative and qualitative experiments demonstrate that the proposed M3SR outperforms existing state-of-the-art methods while incurring a lower computational cost.

Problem

• spectral reconstruction
• Mamba architecture
• multi-scale feature extraction
• spatial perception
• hyperspectral images

Innovation

• Mamba architecture
• multi-scale feature extraction
• multi-perceptual fusion
• spectral reconstruction
• hyperspectral imaging