SpectraLift: Physics-Guided Spectral-Inversion Network for Self-Supervised Hyperspectral Image Super-Resolution

2025-07-17

AI Summary

Low spatial resolution of hyperspectral images (HSI) limits their utility in remote sensing and biomedical applications. Existing HSI–multispectral image (MSI) fusion methods rely on either difficult-to-obtain point spread functions (PSFs) or ground-truth high-resolution HSIs (HR-HSIs), hindering practical deployment. This paper proposes SpectraLift, a fully self-supervised framework that synthesizes training signals solely from the known spectral response functions of the MSI sensor—requiring no external calibration or HR-HSI ground truth. It employs a lightweight per-pixel MLP, trained end-to-end via an ℓ₁ spectral reconstruction loss, directly mapping high-resolution MSIs to high-resolution HSIs. The model converges within minutes and consistently outperforms state-of-the-art methods across PSNR, SAM, SSIM, and RMSE metrics. Moreover, it exhibits strong robustness to spatial blur and resolution mismatches, demonstrating cross-device and cross-resolution generalization capability.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

High-spatial-resolution hyperspectral images (HSI) are essential for applications such as remote sensing and medical imaging, yet HSI sensors inherently trade spatial detail for spectral richness. Fusing high-spatial-resolution multispectral images (HR-MSI) with low-spatial-resolution hyperspectral images (LR-HSI) is a promising route to recover fine spatial structures without sacrificing spectral fidelity. Most state-of-the-art methods for HSI-MSI fusion demand point spread function (PSF) calibration or ground truth high resolution HSI (HR-HSI), both of which are impractical to obtain in real world settings. We present SpectraLift, a fully self-supervised framework that fuses LR-HSI and HR-MSI inputs using only the MSI's Spectral Response Function (SRF). SpectraLift trains a lightweight per-pixel multi-layer perceptron (MLP) network using ($i$)~a synthetic low-spatial-resolution multispectral image (LR-MSI) obtained by applying the SRF to the LR-HSI as input, ($ii$)~the LR-HSI as the output, and ($iii$)~an $ell_1$ spectral reconstruction loss between the estimated and true LR-HSI as the optimization objective. At inference, SpectraLift uses the trained network to map the HR-MSI pixel-wise into a HR-HSI estimate. SpectraLift converges in minutes, is agnostic to spatial blur and resolution, and outperforms state-of-the-art methods on PSNR, SAM, SSIM, and RMSE benchmarks.

Problem

• Fuse LR-HSI and HR-MSI without PSF or ground truth HR-HSI
• Achieve self-supervised HSI super-resolution using only SRF
• Recover high-spatial-resolution HSI without spectral fidelity loss

Innovation

• Self-supervised fusion of LR-HSI and HR-MSI
• Uses only MSI's Spectral Response Function
• Lightweight per-pixel MLP network training