Broadband Hyperspectral 3D Imaging using Dispersed Structured Light

2026-05-25

AI Summary

This work addresses the challenge of simultaneously acquiring high-fidelity reflectance spectra and dense 3D geometry across the broad 450–1500 nm spectral range, a task hindered by the complexity of conventional multi-sensor hyperspectral 3D imaging systems. The authors propose a novel single-spectrometer-based broadband hyperspectral 3D imaging method that, for the first time, extends dispersive structured light illumination to the full visible–near-infrared–shortwave infrared spectrum. By integrating stereo vision with a joint reconstruction model, the approach enables simultaneous, dense estimation of hyperspectral reflectance and depth. Experimental results on real-world scenes demonstrate an average spectral angle error of 0.13 radians, a root-mean-square error of 0.03, and a mean depth error of 4.5 mm. The method successfully facilitates applications including metamer identification, subsurface imaging, and subcutaneous blood vessel visualization.

Abstract

Hyperspectral 3D imaging enables the capture of dense spectral information and scene geometry but has traditionally been confined to narrow spectral windows, typically the visible range. In this work, we introduce a broadband hyperspectral 3D imaging (BH3D) method to extend this capability across the full visible-near-infrared and short-wavelength infrared (SWIR) spectrum (450-1500 nm). This broad coverage is critical as it captures complementary physical cues: visible wavelengths reveal surface appearance, while SWIR bands provide insight into subsurface properties and material composition. However, realizing BH3D is challenging due to fundamental sensor constraints between visible-spectrum silicon and SWIR-spectrum InGaAs sensors, which necessitate complex multi-spectrograph designs. Here we propose a single-spectrograph BH3D system, using a stereo setup comprising visible and SWIR cameras, that reconstructs dense broadband hyperspectral reflectance together with accurate 3D geometry. Our key idea is to extend dispersed structured light to the broadband regime using a single spectrograph. We model the image formation of broadband dispersed structured light, and estimate hyperspectral reflectance and depth. We validate our approach on diverse real-world scenes, demonstrating accurate reconstruction with a mean spectral angle mapper of 0.13 rad, root mean square error of 0.03, and mean depth error of 4.5 mm. We further demonstrate identifying metameric materials, performing imaging through opaque layers, uncovering hidden features on banknotes, and revealing blood vessels.

Problem

• hyperspectral imaging
• 3D reconstruction
• broadband spectrum
• SWIR
• structured light

Innovation

• broadband hyperspectral imaging
• dispersed structured light
• 3D reconstruction
• SWIR imaging
• single-spectrograph system