HS-3D-NeRF: 3D Surface and Hyperspectral Reconstruction From Stationary Hyperspectral Images Using Multi-Channel NeRFs

2026-02-18

AI Summary

This study addresses the challenge of efficiently integrating hyperspectral imaging with 3D reconstruction in automated agricultural phenotyping, a task often hindered by complex hardware setups or multi-view moving cameras that compromise throughput and reproducibility. The authors propose a fixed-camera, multi-channel Neural Radiance Field (NeRF) framework that captures multi-view hyperspectral images of rotating objects within a custom diffuse illumination chamber, leveraging ArUco markers for pose estimation and spatial alignment. This work presents the first application of multi-channel NeRF to fixed-camera hyperspectral 3D reconstruction, introducing a two-stage training strategy to decouple geometry initialization from radiance field optimization and a composite spectral loss function to enhance cross-band consistency. Experiments on three agricultural products demonstrate high spatial reconstruction accuracy and excellent spectral fidelity across visible to near-infrared wavelengths, highlighting its suitability for high-throughput agricultural quality inspection.

Technology Category

• Computer Vision: 3D Computer Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Multi-instance/Multi-view Learning

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Economics, Online Markets and Human Computation: Architectures and workflows that use LLMs for crowd work

Abstract

Advances in hyperspectral imaging (HSI) and 3D reconstruction have enabled accurate, high-throughput characterization of agricultural produce quality and plant phenotypes, both essential for advancing agricultural sustainability and breeding programs. HSI captures detailed biochemical features of produce, while 3D geometric data substantially improves morphological analysis. However, integrating these two modalities at scale remains challenging, as conventional approaches involve complex hardware setups incompatible with automated phenotyping systems. Recent advances in neural radiance fields (NeRF) offer computationally efficient 3D reconstruction but typically require moving-camera setups, limiting throughput and reproducibility in standard indoor agricultural environments. To address these challenges, we introduce HSI-SC-NeRF, a stationary-camera multi-channel NeRF framework for high-throughput hyperspectral 3D reconstruction targeting postharvest inspection of agricultural produce. Multi-view hyperspectral data is captured using a stationary camera while the object rotates within a custom-built Teflon imaging chamber providing diffuse, uniform illumination. Object poses are estimated via ArUco calibration markers and transformed to the camera frame of reference through simulated pose transformations, enabling standard NeRF training on stationary-camera data. A multi-channel NeRF formulation optimizes reconstruction across all hyperspectral bands jointly using a composite spectral loss, supported by a two-stage training protocol that decouples geometric initialization from radiometric refinement. Experiments on three agricultural produce samples demonstrate high spatial reconstruction accuracy and strong spectral fidelity across the visible and near-infrared spectrum, confirming the suitability of HSI-SC-NeRF for integration into automated agricultural workflows.

Problem

• hyperspectral imaging
• 3D reconstruction
• stationary camera
• agricultural phenotyping
• NeRF

Innovation

• multi-channel NeRF
• stationary-camera hyperspectral imaging
• 3D surface reconstruction
• spectral fidelity
• two-stage training protocol