Lumosaic: Hyperspectral Video via Active Illumination and Coded-Exposure Pixels

2026-02-25

AI Summary

This work addresses the limitations of existing snapshot hyperspectral imaging systems in dynamic scenes, where low photon efficiency and motion blur hinder simultaneous spectral fidelity and temporal consistency. The authors propose a novel paradigm that co-designs active illumination with a coded exposure pixel (CEP) camera: by integrating a narrowband LED array and pixel-level exposure control, spatial, temporal, and spectral information are jointly encoded within a single frame and subsequently reconstructed via a learning-based algorithm to recover hyperspectral video. This approach achieves the first co-optimization of active illumination and CEP, overcoming the spectral-temporal trade-off inherent in conventional passive systems. Experiments demonstrate real-time reconstruction of 30 fps, VGA-resolution hyperspectral video with 31 spectral channels (400–700 nm), significantly outperforming state-of-the-art snapshot methods on both synthetic and real-world data involving complex materials and motion.

Technology Category

• Computer Vision: Computational Photography, Image & Video Synthesis
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Machine Learning: Active Learning

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

We present Lumosaic, a compact active hyperspectral video system designed for real-time capture of dynamic scenes. Our approach combines a narrowband LED array with a coded-exposure-pixel (CEP) camera capable of high-speed, per-pixel exposure control, enabling joint encoding of scene information across space, time, and wavelength within each video frame. Unlike passive snapshot systems that divide light across multiple spectral channels simultaneously and assume no motion during a frame's exposure, Lumosaic actively synchronizes illumination and pixel-wise exposure, improving photon utilization and preserving spectral fidelity under motion. A learning-based reconstruction pipeline then recovers 31-channel hyperspectral (400-700 nm) video at 30 fps and VGA resolution, producing temporally coherent and spectrally accurate reconstructions. Experiments on synthetic and real data demonstrate that Lumosaic significantly improves reconstruction fidelity and temporal stability over existing snapshot hyperspectral imaging systems, enabling robust hyperspectral video across diverse materials and motion conditions.

Problem

• hyperspectral video
• dynamic scenes
• motion artifacts
• photon efficiency
• spectral fidelity

Innovation

• active illumination
• coded-exposure pixels
• hyperspectral video
• learning-based reconstruction
• real-time spectral imaging