Interactive Spatial-Frequency Fusion Mamba for Multi-Modal Image Fusion

2026-02-04

AI Summary

This work addresses the limited interaction between spatial and frequency information in existing multimodal image fusion methods, which often results in insufficient detail preservation and constrained cross-modal complementarity. To overcome this, we propose an interactive spatial-frequency fusion Mamba framework that introduces, for the first time, a dynamic mechanism wherein frequency-domain information adaptively guides spatial feature fusion. Leveraging the Mamba state space model, our approach efficiently captures long-range dependencies with linear computational complexity. The architecture integrates modality-specific extractors, multi-scale frequency decomposition, and an adaptive fusion module to effectively harness complementary cross-modal information. Extensive experiments demonstrate that our method significantly outperforms current state-of-the-art approaches across six benchmark datasets, achieving notable improvements in both detail retention and overall information fidelity.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

Multi-Modal Image Fusion (MMIF) aims to combine images from different modalities to produce fused images, retaining texture details and preserving significant information. Recently, some MMIF methods incorporate frequency domain information to enhance spatial features. However, these methods typically rely on simple serial or parallel spatial-frequency fusion without interaction. In this paper, we propose a novel Interactive Spatial-Frequency Fusion Mamba (ISFM) framework for MMIF. Specifically, we begin with a Modality-Specific Extractor (MSE) to extract features from different modalities. It models long-range dependencies across the image with linear computational complexity. To effectively leverage frequency information, we then propose a Multi-scale Frequency Fusion (MFF). It adaptively integrates low-frequency and high-frequency components across multiple scales, enabling robust representations of frequency features. More importantly, we further propose an Interactive Spatial-Frequency Fusion (ISF). It incorporates frequency features to guide spatial features across modalities, enhancing complementary representations. Extensive experiments are conducted on six MMIF datasets. The experimental results demonstrate that our ISFM can achieve better performances than other state-of-the-art methods. The source code is available at https://github.com/Namn23/ISFM.

Problem

• Multi-Modal Image Fusion
• Spatial-Frequency Fusion
• Image Fusion
• Frequency Domain
• Feature Interaction

Innovation

• Interactive Spatial-Frequency Fusion
• Mamba
• Multi-Modal Image Fusion
• Multi-scale Frequency Fusion
• Modality-Specific Extractor