🤖 AI Summary
This study addresses the challenge of hyperspectral image fusion under unregistered conditions and unknown camera response functions by proposing a supervised framework based on permutation-invariant Gram matrices. Leveraging the permutation invariance of abundance map Gram matrices, the method integrates a residual spectral super-resolution network with a random pixel permutation validation strategy to enable direct RGB-to-high-resolution hyperspectral reconstruction without spatial registration, camera response functions, or paired data. Experiments demonstrate that this approach achieves accuracy comparable to conventional assumption-dependent methods across multiple scene benchmarks while maintaining robustness when such assumptions fail. Notably, its performance advantage stems from fundamental principled innovation rather than loss function tuning, offering a reliable solution for real-world scenarios where standard calibration and alignment prerequisites are unavailable.
📝 Abstract
Reconstructing a spatially and spectrally high-resolution hyperspectral image (HR-HSI) from a low-resolution HSI (LR-HSI) and a high-resolution RGB image (HR-RGB) usually assumes precise registration and a known camera response function (CRF). Both assumptions are difficult to satisfy with different sensors. We remove both through a permutation-invariant supervision principle: the Gram matrix of an unmixed abundance map depends on shared material composition but not on pixel ordering. Matching abundance Gram matrices therefore allows RGB-to-HSI mapping to be learned without spatial correspondence and without a predefined CRF. Under a full random permutation of HR-RGB pixels, a state-of-the-art fusion method collapses, whereas our reconstruction is unchanged after inverse reindexing for evaluation. Building on this principle, a residual spectral super-resolution function maps HR-RGB directly to HR-HSI without registration, known CRF, or paired supervision. Across indoor, natural-scene, and remote-sensing benchmarks, the method achieves accuracy comparable to approaches that require these assumptions while remaining robust when they are violated. Loss ablations further show that reconstruction accuracy is largely insensitive to the specific discrepancy used to match the Gram matrices, indicating that performance arises primarily from the permutation-invariant principle rather than loss tuning.