Non-line-of-sight imaging with arbitrary relay surface geometries via 3D Gaussian Transient Rendering

2026-06-19

AI Summary

This work addresses the limitations of existing non-line-of-sight (NLOS) imaging methods, which typically rely on planar relay walls and dense sampling, rendering them ill-suited for real-world scenarios with spatial constraints and arbitrarily shaped relay surfaces. The authors propose an end-to-end reconstruction framework that requires no prior knowledge of the relay surface geometry. By representing the hidden scene with 3D Gaussian primitives and integrating a differentiable transient rendering model, the method directly optimizes reconstructions from measured time-of-flight (ToF) data. It is the first approach to support arbitrarily shaped relay surfaces and is compatible with both confocal and non-confocal configurations. Under sparse sampling conditions, it achieves state-of-the-art reconstruction quality on both public and newly introduced datasets, significantly outperforming current methods.

Abstract

Imaging objects hidden outside the direct line of sight expands the effective field of view and is critical for applications such as autonomous driving and robotic perception. Despite impressive progress in time-of-flight (ToF)-based non-line-of-sight (NLOS) imaging, real-world deployment remains challenging because practical measurements are often collected over spatially limited, arbitrarily shaped relay regions-conditions that violate the planar-wall and dense-sampling assumptions made by most existing methods. To address these limitations, we propose a LOS-guided NLOS imaging pipeline that imposes no geometric assumptions on the relay surface and naturally supports both confocal and non-confocal configurations. Our method represents the hidden scene using 3D Gaussian primitives and couples them with an efficient, differentiable transient rendering model, enabling end-to-end optimization directly from measured transients. We validate our approach on real-world measurements from both a public dataset and a custom-built capture system. Across settings, our method achieves state-of-the-art reconstruction fidelity under spatially limited, sparsely sampled conditions, and significantly outperforms existing methods on complex, arbitrary relay surface geometries.

Problem

• non-line-of-sight imaging
• arbitrary relay surface
• spatially limited sampling
• time-of-flight
• real-world deployment

Innovation

• non-line-of-sight imaging
• 3D Gaussian representation
• transient rendering
• arbitrary relay surfaces
• differentiable rendering