Taming Flow Matching with Unbalanced Optimal Transport into Fast Pansharpening

2025-03-19

AI Summary

This work addresses the high computational cost and inherent quality-efficiency trade-off in multi-step sampling of score-based stochastic differential equation (SDE) diffusion models for remote sensing image pansharpening. We propose the first method embedding the dual formulation of unbalanced optimal transport (UOT) into the flow matching framework: edge constraints are relaxed to accommodate the intrinsic spectral-spatial discrepancy in remote sensing data, and joint regularization is introduced to preserve both spectral fidelity and spatial detail. Our approach employs a simulation-free training strategy and enables single-step, deterministic, high-fidelity inference. On multiple benchmark datasets, it matches or surpasses state-of-the-art regression- and diffusion-based methods in reconstruction quality while accelerating inference by over three orders of magnitude. This substantially alleviates the computational-quality trade-off. The source code is publicly available.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Pansharpening, a pivotal task in remote sensing for fusing high-resolution panchromatic and multispectral imagery, has garnered significant research interest. Recent advancements employing diffusion models based on stochastic differential equations (SDEs) have demonstrated state-of-the-art performance. However, the inherent multi-step sampling process of SDEs imposes substantial computational overhead, hindering practical deployment. While existing methods adopt efficient samplers, knowledge distillation, or retraining to reduce sampling steps (e.g., from 1,000 to fewer steps), such approaches often compromise fusion quality. In this work, we propose the Optimal Transport Flow Matching (OTFM) framework, which integrates the dual formulation of unbalanced optimal transport (UOT) to achieve one-step, high-quality pansharpening. Unlike conventional OT formulations that enforce rigid distribution alignment, UOT relaxes marginal constraints to enhance modeling flexibility, accommodating the intrinsic spectral and spatial disparities in remote sensing data. Furthermore, we incorporate task-specific regularization into the UOT objective, enhancing the robustness of the flow model. The OTFM framework enables simulation-free training and single-step inference while maintaining strict adherence to pansharpening constraints. Experimental evaluations across multiple datasets demonstrate that OTFM matches or exceeds the performance of previous regression-based models and leading diffusion-based methods while only needing one sampling step. Codes are available at https://github.com/294coder/PAN-OTFM.

Problem

• Reduces computational overhead in pansharpening
• Achieves high-quality fusion in one step
• Enhances modeling flexibility with unbalanced optimal transport

Innovation

• Optimal Transport Flow Matching framework
• Unbalanced Optimal Transport integration
• Single-step inference for pansharpening