Towards 3D Scene Understanding of Gas Plumes in LWIR Hyperspectral Images Using Neural Radiance Fields

2026-03-05

AI Summary

This work addresses the challenge of three-dimensional gas plume modeling and detection in sparse long-wave infrared (LWIR) hyperspectral imagery by proposing a neural radiance field (NeRF) approach that integrates hyperspectral information with sparse-view constraints. Built upon the Mip-NeRF architecture, the method combines hyperspectral NeRF and sparse-view NeRF components and employs an adaptive weighted mean squared error (MSE) loss function. Using only 30 training images—50% fewer than conventional approaches—it achieves high-fidelity novel view synthesis with an average peak signal-to-noise ratio (PSNR) of 39.8 dB. Coupled with an adaptive coherence estimator for gas detection, the framework attains an area under the curve (AUC) of 0.821 on the DIRSIG synthetic dataset, significantly enhancing the reconstruction and identification of 3D gas plumes under sparse observational conditions.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Multi-instance/Multi-view Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Hyperspectral images (HSI) have many applications, ranging from environmental monitoring to national security, and can be used for material detection and identification. Longwave infrared (LWIR) HSI can be used for gas plume detection and analysis. Oftentimes, only a few images of a scene of interest are available and are analyzed individually. The ability to combine information from multiple images into a single, cohesive representation could enhance analysis by providing more context on the scene's geometry and spectral properties. Neural radiance fields (NeRFs) create a latent neural representation of volumetric scene properties that enable novel-view rendering and geometry reconstruction, offering a promising avenue for hyperspectral 3D scene reconstruction. We explore the possibility of using NeRFs to create 3D scene reconstructions from LWIR HSI and demonstrate that the model can be used for the basic downstream analysis task of gas plume detection. The physics-based DIRSIG software suite was used to generate a synthetic multi-view LWIR HSI dataset of a simple facility with a strong sulfur hexafluoride gas plume. Our method, built on the standard Mip-NeRF architecture, combines state-of-the-art methods for hyperspectral NeRFs and sparse-view NeRFs, along with a novel adaptive weighted MSE loss. Our final NeRF method requires around 50% fewer training images than the standard Mip-NeRF and achieves an average PSNR of 39.8 dB with as few as 30 training images. Gas plume detection applied to NeRF-rendered test images using the adaptive coherence estimator achieves an average AUC of 0.821 when compared with detection masks generated from ground-truth test images.

Problem

• 3D scene understanding
• gas plumes
• LWIR hyperspectral images
• neural radiance fields
• sparse-view reconstruction

Innovation

• Neural Radiance Fields
• LWIR Hyperspectral Imaging
• Gas Plume Detection
• Sparse-View Reconstruction
• Adaptive Weighted Loss