Spectral Super-Resolution Neural Operator with Atmospheric Radiative Transfer Prior

2025-11-21

AI Summary

To address spectral distortion—particularly in atmospheric absorption bands—caused by physical inconsistency in multispectral-to-hyperspectral image (MSI-to-HSI) reconstruction, this paper proposes a three-stage spectral super-resolution framework integrating atmospheric radiative transfer priors with data-driven neural operators. Methodologically, it (1) incorporates a differentiable atmospheric radiative transfer model as a physics-based constraint; (2) introduces a guided matrix projection (GMP) mechanism, with theoretical guarantees of optimality for continuous-spectrum reconstruction and zero-shot extrapolation; and (3) designs a U-shaped spectral-aware convolutional (SAC) neural operator that jointly performs upsampling, spectral reconstruction, and physics-informed refinement. Experiments demonstrate substantial improvements in spectral fidelity and spatial-spectral consistency, effective suppression of color bias, and superior performance over state-of-the-art methods on both real-world data and cross-sensor generalization tasks.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Feature Construction/Reformulation
• Planning, Routing, and Scheduling: Replanning and Plan Repair

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Security and Privacy: Data transparency and provenance

Abstract

Spectral super-resolution (SSR) aims to reconstruct hyperspectral images (HSIs) from multispectral observations, with broad applications in remote sensing. Data-driven methods are widely used, but they often overlook physical principles, leading to unrealistic spectra, particularly in atmosphere-affected bands. To address this challenge, we propose the Spectral Super-Resolution Neural Operator (SSRNO), which incorporates atmospheric radiative transfer (ART) prior into the data-driven procedure, yielding more physically consistent predictions. The proposed SSRNO framework consists of three stages: upsampling, reconstruction, and refinement. In the upsampling stage, we leverage prior information to expand the input multispectral image, producing a physically plausible hyperspectral estimate. Subsequently, we utilize a neural operator in the reconstruction stage to learn a continuous mapping across the spectral domain. Finally, the refinement stage imposes a hard constraint on the output HSI to eliminate color distortion. The upsampling and refinement stages are implemented via the proposed guidance matrix projection (GMP) method, and the reconstruction neural operator adopts U-shaped spectral-aware convolution (SAC) layers to capture multi-scale features. Moreover, we theoretically demonstrate the optimality of the GMP method. With the neural operator and ART priors, SSRNO also achieves continuous spectral reconstruction and zero-shot extrapolation. Various experiments validate the effectiveness and generalization ability of the proposed approach.

Problem

• Reconstruct hyperspectral images from multispectral observations
• Incorporate atmospheric radiative transfer prior into data-driven methods
• Eliminate color distortion and achieve physically consistent spectral predictions

Innovation

• Incorporates atmospheric radiative transfer prior into neural network
• Uses guidance matrix projection for upsampling and refinement stages
• Adopts spectral-aware convolution layers for multi-scale feature capture