Grayscale to Hyperspectral at Any Resolution Using a Phase-Only Lens

2024-12-03

arXiv.org

AI Summary

This work addresses the ill-posed inverse problem of reconstructing an H×W×31 hyperspectral image from a single grayscale snapshot (H×W) captured by a color-filterless panchromatic sensor through a single-phase-modulation diffractive lens. We propose the first optical point-spread-function (PSF)-guided conditional denoising diffusion model. Our method integrates physical priors with generative modeling: the analytically computable local PSF conditions the diffusion process, and shift-invariant patch-wise processing enables arbitrary-resolution reconstruction—all without spectral filtering and using only one input frame. Key contributions include: (1) empirical validation that local optical cues alone suffice for full-spectrum reconstruction; (2) high-fidelity, high-resolution recovery at patch sizes down to the PSF scale; and (3) the first pixel-level uncertainty quantification in this setting, where predicted confidence exhibits strong correlation with actual reconstruction error.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Search and Optimization: Non-convex Optimization
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Security and Privacy: Data transparency and provenance
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs

Abstract

We consider the problem of reconstructing a HxWx31 hyperspectral image from a HxW grayscale snapshot measurement that is captured using only a single diffractive optic and a filterless panchromatic photosensor. This problem is severely ill-posed, but we present the first model that produces high-quality results. We make efficient use of limited data by training a conditional denoising diffusion model that operates on small patches in a shift-invariant manner. During inference, we synchronize per-patch hyperspectral predictions using guidance derived from the optical point spread function. Surprisingly, our experiments reveal that patch sizes as small as the PSFs support achieve excellent results, and they show that local optical cues are sufficient to capture full spectral information. Moreover, by drawing multiple samples, our model provides per-pixel uncertainty estimates that strongly correlate with reconstruction error. Our work lays the foundation for a new class of high-resolution snapshot hyperspectral imagers that are compact and light-efficient.

Problem

• Reconstruct hyperspectral image from grayscale snapshot
• Use single diffractive optic and filterless sensor
• Achieve high-resolution with compact light-efficient design

Innovation

• Uses phase-only lens for hyperspectral reconstruction
• Employs conditional denoising diffusion model
• Synchronizes predictions via optical PSF guidance