Two-Stage Random Alternation Framework for Zero-Shot Pansharpening

2025-05-10

AI Summary

To address the bottleneck of requiring large-scale high-resolution labeled data in full-resolution remote sensing image fusion, this paper proposes a zero-shot pansharpening method that requires neither prior knowledge nor extensive training datasets. The method introduces a two-stage stochastic alternating optimization framework: Stage I employs degradation-aware modeling (DAM) and a warm-up strategy to enhance the robustness of initial reconstruction; Stage II implicitly learns the degradation process in an unsupervised manner by stochastically alternating optimization over low-quality observations and a single pair of full-resolution inputs. Crucially, the approach strictly enforces physical imaging constraints, enabling end-to-end zero-shot learning. Evaluated on real-world data, it consistently outperforms state-of-the-art methods, achieving average improvements of 1.2 dB in PSNR and 0.015 in SSIM. It further demonstrates superior visual fidelity, generalization capability, and practical deployability—being lightweight and plug-and-play.

Technology Category

• Computer Vision: Learning & Optimization for CV
• Search and Optimization: Learning to Search
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments

Abstract

In recent years, pansharpening has seen rapid advancements with deep learning methods, which have demonstrated impressive fusion quality. However, the challenge of acquiring real high-resolution images limits the practical applicability of these methods. To address this, we propose a two-stage random alternating framework (TRA-PAN) that effectively integrates strong supervision constraints from reduced-resolution images with the physical characteristics of full-resolution images. The first stage introduces a pre-training procedure, which includes Degradation-Aware Modeling (DAM) to capture spatial-spectral degradation mappings, alongside a warm-up procedure designed to reduce training time and mitigate the negative effects of reduced-resolution data. In the second stage, Random Alternation Optimization (RAO) is employed, where random alternating training leverages the strengths of both reduced- and full-resolution images, further optimizing the fusion model. By primarily relying on full-resolution images, our method enables zero-shot training with just a single image pair, obviating the need for large datasets. Experimental results demonstrate that TRA-PAN outperforms state-of-the-art (SOTA) methods in both quantitative metrics and visual quality in real-world scenarios, highlighting its strong practical applicability.

Problem

• Overcoming limited real high-resolution image acquisition for pansharpening
• Integrating reduced- and full-resolution image constraints effectively
• Enabling zero-shot training with minimal image pairs

Innovation

• Two-stage random alternating framework (TRA-PAN)
• Degradation-Aware Modeling (DAM) for spatial-spectral mappings
• Random Alternation Optimization (RAO) for resolution adaptation