MFmamba: A Multi-function Network for Panchromatic Image Resolution Restoration Based on State-Space Model

2025-11-24

AI Summary

To address the challenge in remote sensing of jointly enhancing both spatial and spectral resolution from a single high-resolution panchromatic (PAN) image, this paper proposes MFmamba, a multi-functional state-space network. Methodologically, MFmamba is the first to introduce the Mamba architecture into single-image, multi-task remote sensing reconstruction. It adopts a lightweight UNet++-based framework integrating three novel components: Mamba-based upsampling blocks, dual-pool attention modules, and multi-scale hybrid cross blocks—enabling unified super-resolution, spectral recovery, and joint pansharpening. Unlike conventional methods requiring multiple inputs (e.g., PAN + multispectral) or targeting only one objective, MFmamba operates solely on a single PAN image, significantly improving reconstruction accuracy and generalizability. Extensive experiments demonstrate that MFmamba outperforms state-of-the-art approaches in both quantitative metrics (PSNR/SSIM) and visual quality.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multi-instance/Multi-view Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Remote sensing images are becoming increasingly widespread in military, earth resource exploration. Because of the limitation of a single sensor, we can obtain high spatial resolution grayscale panchromatic (PAN) images and low spatial resolution color multispectral (MS) images. Therefore, an important issue is to obtain a color image with high spatial resolution when there is only a PAN image at the input. The existing methods improve spatial resolution using super-resolution (SR) technology and spectral recovery using colorization technology. However, the SR technique cannot improve the spectral resolution, and the colorization technique cannot improve the spatial resolution. Moreover, the pansharpening method needs two registered inputs and can not achieve SR. As a result, an integrated approach is expected. To solve the above problems, we designed a novel multi-function model (MFmamba) to realize the tasks of SR, spectral recovery, joint SR and spectral recovery through three different inputs. Firstly, MFmamba utilizes UNet++ as the backbone, and a Mamba Upsample Block (MUB) is combined with UNet++. Secondly, a Dual Pool Attention (DPA) is designed to replace the skip connection in UNet++. Finally, a Multi-scale Hybrid Cross Block (MHCB) is proposed for initial feature extraction. Many experiments show that MFmamba is competitive in evaluation metrics and visual results and performs well in the three tasks when only the input PAN image is used.

Problem

• Restoring high-resolution color images from single panchromatic inputs
• Integrating super-resolution and spectral recovery in one model
• Overcoming limitations of separate SR and colorization techniques

Innovation

• MFmamba uses UNet++ backbone with Mamba Upsample Block
• Dual Pool Attention replaces skip connections in UNet++
• Multi-scale Hybrid Cross Block enables initial feature extraction