Enhancing Unregistered Hyperspectral Image Super-Resolution via Unmixing-based Abundance Fusion Learning

2026-03-09

AI Summary

This work addresses the challenge of performance degradation in unregistered hyperspectral image super-resolution caused by misalignment between the reference and target images. To overcome this, we propose a disentanglement-based abundance fusion learning framework that decouples spatial and spectral information through spectral unmixing. A coarse-to-fine subpixel-level deformable aggregation module is introduced to align features from unregistered high-resolution reference images, while a spatial-channel cross-attention mechanism combined with a dynamic gating strategy enables adaptive fusion. By innovatively integrating singular value decomposition–based unmixing, deformable alignment, and modulated fusion into a unified architecture, the method effectively mitigates the adverse effects of misregistration. Extensive experiments on both simulated and real-world datasets demonstrate that our approach consistently achieves state-of-the-art performance, significantly outperforming existing methods.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Unregistered hyperspectral image (HSI) super-resolution (SR) typically aims to enhance a low-resolution HSI using an unregistered high-resolution reference image. In this paper, we propose an unmixing-based fusion framework that decouples spatial-spectral information to simultaneously mitigate the impact of unregistered fusion and enhance the learnability of SR models. Specifically, we first utilize singular value decomposition for initial spectral unmixing, preserving the original endmembers while dedicating the subsequent network to enhancing the initial abundance map. To leverage the spatial texture of the unregistered reference, we introduce a coarse-to-fine deformable aggregation module, which first estimates a pixel-level flow and a similarity map using a coarse pyramid predictor. It further performs fine sub-pixel refinement to achieve deformable aggregation of the reference features. The aggregative features are then refined via a series of spatial-channel abundance cross-attention blocks. Furthermore, a spatial-channel modulated fusion module is presented to merge encoder-decoder features using dynamic gating weights, yielding a high-quality, high-resolution HSI. Experimental results on simulated and real datasets confirm that our proposed method achieves state-of-the-art super-resolution performance. The code will be available at https://github.com/yingkai-zhang/UAFL.

Problem

• unregistered hyperspectral image
• super-resolution
• image fusion
• spectral unmixing
• abundance map

Innovation

• unmixing-based fusion
• unregistered hyperspectral super-resolution
• deformable aggregation
• spatial-channel cross-attention
• dynamic gating fusion