A General Adaptive Dual-level Weighting Mechanism for Remote Sensing Pansharpening

2025-03-17

AI Summary

To address insufficient modeling of feature heterogeneity and difficulty in suppressing intra-channel and inter-layer redundancy in remote sensing pan-sharpening, this paper proposes a Covariance-Driven Adaptive Dual-level Weighting Mechanism (ADWM). ADWM introduces a novel covariance-aware dual-level weighting paradigm: Intra-Feature Weighting (IFW) suppresses intra-layer redundancy via channel-wise covariance estimation, while Cross-Layer Weighting (CFW) dynamically modulates inter-layer feature contributions using cross-layer covariance. It is the first method to deeply integrate covariance matrix modeling with nonlinear weight mapping, enabling adaptive feature refinement across channels and layers. ADWM is plug-and-play and compatible with mainstream network architectures. Extensive experiments on multiple remote sensing datasets demonstrate significant improvements over state-of-the-art methods, with consistent gains in PSNR (+0.12–0.38 dB) and SSIM (+0.003–0.012). The open-sourced implementation has been widely adopted in the community.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Adversarial Learning & Robustness
• Cognitive Modeling & Cognitive Systems: Adaptive Behavior

Application Category

• Responsible Web: Data and user privacy-enhancing technologies for the Web
• Web Mining and Content Analysis: Large pretrained models with web data
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Currently, deep learning-based methods for remote sensing pansharpening have advanced rapidly. However, many existing methods struggle to fully leverage feature heterogeneity and redundancy, thereby limiting their effectiveness. We use the covariance matrix to model the feature heterogeneity and redundancy and propose Correlation-Aware Covariance Weighting (CACW) to adjust them. CACW captures these correlations through the covariance matrix, which is then processed by a nonlinear function to generate weights for adjustment. Building upon CACW, we introduce a general adaptive dual-level weighting mechanism (ADWM) to address these challenges from two key perspectives, enhancing a wide range of existing deep-learning methods. First, Intra-Feature Weighting (IFW) evaluates correlations among channels within each feature to reduce redundancy and enhance unique information. Second, Cross-Feature Weighting (CFW) adjusts contributions across layers based on inter-layer correlations, refining the final output. Extensive experiments demonstrate the superior performance of ADWM compared to recent state-of-the-art (SOTA) methods. Furthermore, we validate the effectiveness of our approach through generality experiments, redundancy visualization, comparison experiments, key variables and complexity analysis, and ablation studies. Our code is available at https://github.com/Jie-1203/ADWM.

Problem

• Addresses feature heterogeneity and redundancy in remote sensing pansharpening.
• Proposes Correlation-Aware Covariance Weighting (CACW) for feature adjustment.
• Introduces adaptive dual-level weighting mechanism (ADWM) to enhance deep-learning methods.

Innovation

• Uses Correlation-Aware Covariance Weighting (CACW)
• Implements Intra-Feature Weighting (IFW)
• Applies Cross-Feature Weighting (CFW)