Bi^2MAC: Bimodal Bi-Adaptive Mask-Aware Convolution for Remote Sensing Pansharpening

2025-12-09

AI Summary

Existing deep learning methods for remote sensing pan-sharpening struggle to model spatial heterogeneity, while current adaptive convolution approaches suffer from high computational overhead and limited receptive-field awareness. To address these issues, this paper proposes a Dual-Modality Dual-Adaptive Mask-Percipient Convolution (DAMPC) mechanism. DAMPC introduces, for the first time, a soft–hard dual-mask co-generation strategy that dynamically identifies heterogeneous versus redundant regions and routes them to fine-grained or lightweight branches, respectively. By integrating lightweight mask generation, adaptive convolution, feature modulation, and branch routing, DAMPC achieves a balanced trade-off between global efficiency and local precision. Evaluated on multiple benchmark datasets, DAMPC achieves state-of-the-art performance with 32%–57% fewer parameters and 21%–39% reduced training time—making it the most computationally efficient adaptive convolution-based pan-sharpening model to date.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Security and Privacy: Privacy-enhancing technologies
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

Pansharpening aims to fuse a high-resolution panchromatic (PAN) image with a low-resolution multispectral (LRMS) image to generate a high-resolution multispectral image (HRMS). Conventional deep learning-based methods are inherently limited in their ability to adapt to regional heterogeneity within feature representations. Although various adaptive convolution methods have been proposed to address this limitation, they often suffer from excessive computational costs and a limited ability to capture heterogeneous regions in remote sensing images effectively. To overcome these challenges, we propose Bimodal Bi-Adaptive Mask-Aware Convolution (Bi^2MAC), which effectively exploits information from different types of regions while intelligently allocating computational resources. Specifically, we design a lightweight module to generate both soft and hard masks, which are used to modulate the input features preliminarily and to guide different types of regions into separate processing branches, respectively. Redundant features are directed to a compact branch for low-cost global processing. In contrast, heterogeneous features are routed to a focused branch that invests more computational resources for fine-grained modeling. Extensive experiments on multiple benchmark datasets demonstrate that Bi^2MAC achieves state-of-the-art (SOTA) performance while requiring substantially lower training time and parameter counts, and the minimal computational cost among adaptive convolution models.

Problem

• Fuse high-resolution panchromatic with low-resolution multispectral images.
• Adapt to regional heterogeneity in remote sensing feature representations.
• Reduce computational costs while effectively capturing heterogeneous regions.

Innovation

• Bimodal mask generation for feature modulation
• Dual-branch routing for heterogeneous and redundant features
• Lightweight adaptive convolution reducing computational cost