SpectraMorph: Structured Latent Learning for Self-Supervised Hyperspectral Super-Resolution

2025-10-23

AI Summary

Hyperspectral images (HSIs) suffer from low spatial resolution, leading to blurred boundaries and mixed pixels; existing deep learning methods lack interpretability and degrade severely under extremely limited multispectral image (MSI) bands. This paper proposes a physics-guided self-supervised super-resolution framework: leveraging linear spectral unmixing as a physical prior, it establishes an interpretable pipeline comprising endmember extraction, abundance prediction, and spectral-spatial reconstruction, augmented by structured latent-space modeling. Crucially, the framework incorporates MSI sensor spectral response functions to enable label-free self-supervised training. It supports ultra-narrowband or even single-band MSI inputs, achieves highly efficient training (<1 minute), and consistently outperforms state-of-the-art unsupervised and self-supervised methods on both synthetic and real-world data—matching the performance of supervised SOTA approaches.

Technology Category

• Machine Learning: Unsupervised & Self-Supervised Learning
• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Hyperspectral sensors capture dense spectra per pixel but suffer from low spatial resolution, causing blurred boundaries and mixed-pixel effects. Co-registered companion sensors such as multispectral, RGB, or panchromatic cameras provide high-resolution spatial detail, motivating hyperspectral super-resolution through the fusion of hyperspectral and multispectral images (HSI-MSI). Existing deep learning based methods achieve strong performance but rely on opaque regressors that lack interpretability and often fail when the MSI has very few bands. We propose SpectraMorph, a physics-guided self-supervised fusion framework with a structured latent space. Instead of direct regression, SpectraMorph enforces an unmixing bottleneck: endmember signatures are extracted from the low-resolution HSI, and a compact multilayer perceptron predicts abundance-like maps from the MSI. Spectra are reconstructed by linear mixing, with training performed in a self-supervised manner via the MSI sensor's spectral response function. SpectraMorph produces interpretable intermediates, trains in under a minute, and remains robust even with a single-band (pan-chromatic) MSI. Experiments on synthetic and real-world datasets show SpectraMorph consistently outperforming state-of-the-art unsupervised/self-supervised baselines while remaining very competitive against supervised baselines.

Problem

• Enhancing hyperspectral image spatial resolution via fusion
• Addressing interpretability issues in deep learning super-resolution methods
• Maintaining robustness with very few multispectral image bands

Innovation

• Physics-guided self-supervised fusion framework with structured latent space
• Enforces unmixing bottleneck with endmember extraction and abundance prediction
• Trains via spectral response function for robust single-band MSI performance