Problem
Research questions and friction points this paper is trying to address.
Non-Line-of-Sight Imaging
Multiple Light Paths
Computational Efficiency
Passive Non-Line-of-Sight Imaging with Light Transport Modulation
🤖 AI Summary
Existing passive non-line-of-sight (NLOS) imaging methods require training separate models for distinct light transport conditions, suffering from poor generalizability and high computational overhead. To address this, we propose the first unified modeling framework for passive NLOS imaging. Our approach introduces an implicit light transport modulation mechanism: a lightweight encoder coupled with a vector quantization module learns compact representations of environmental light scattering, while multi-scale modulation blocks enable adaptive single-network reconstruction across diverse scenarios. Furthermore, we devise a joint reconstruction-reprojection training paradigm, enforcing geometric reversibility to constrain representation fidelity. Evaluated on a large-scale passive NLOS dataset, our single model generalizes effectively to unseen scattering conditions—achieving a 12.7 dB PSNR improvement over state-of-the-art methods. The implementation is publicly available.
📝 Abstract
Passive non-line-of-sight (NLOS) imaging has witnessed rapid development in recent years, due to its ability to image objects that are out of sight. The light transport condition plays an important role in this task since changing the conditions will lead to different imaging models. Existing learning-based NLOS methods usually train independent models for different light transport conditions, which is computationally inefficient and impairs the practicality of the models. In this work, we propose NLOS-LTM, a novel passive NLOS imaging method that effectively handles multiple light transport conditions with a single network. We achieve this by inferring a latent light transport representation from the projection image and using this representation to modulate the network that reconstructs the hidden image from the projection image. We train a light transport encoder together with a vector quantizer to obtain the light transport representation. To further regulate this representation, we jointly learn both the reconstruction network and the reprojection network during training. A set of light transport modulation blocks is used to modulate the two jointly trained networks in a multi-scale way. Extensive experiments on a large-scale passive NLOS dataset demonstrate the superiority of the proposed method. The code is available at https://github.com/JerryOctopus/NLOS-LTM.
Innovation
Methods, ideas, or system contributions that make the work stand out.
NLOS-LTM
Passive Imaging
Dual-Network Optimization
🔎 Similar Papers
No similar papers found.
