Dual-branch Graph Feature Learning for NLOS Imaging

2025-02-27

AI Summary

Non-line-of-sight (NLOS) imaging faces two major challenges: high computational and memory overhead in voxel-based representation, and difficulty in jointly reconstructing albedo and depth. To address these, we propose a dual-branch graph feature learning framework. First, we introduce graph neural networks (GNNs) into NLOS voxel modeling—leveraging their inherent capability to handle sparse 3D structures. Second, we decouple the albedo and depth reconstruction pathways, eliminating parameter coupling in the loss function. Third, we design a hybrid convolutional-GNN architecture with sparse graph encoding to enable end-to-end joint optimization. Evaluated on both synthetic and real NLOS datasets, our method achieves state-of-the-art performance: it significantly reduces memory footprint while consistently improving reconstruction accuracy across all metrics.

Technology Category

• Computer Vision: Learning & Optimization for CV
• Search and Optimization: Learning to Search
• Machine Learning: Graph-based Machine Learning

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

The domain of non-line-of-sight (NLOS) imaging is advancing rapidly, offering the capability to reveal occluded scenes that are not directly visible. However, contemporary NLOS systems face several significant challenges: (1) The computational and storage requirements are profound due to the inherent three-dimensional grid data structure, which restricts practical application. (2) The simultaneous reconstruction of albedo and depth information requires a delicate balance using hyperparameters in the loss function, rendering the concurrent reconstruction of texture and depth information difficult. This paper introduces the innovative methodology, xnet, which integrates an albedo-focused reconstruction branch dedicated to albedo information recovery and a depth-focused reconstruction branch that extracts geometrical structure, to overcome these obstacles. The dual-branch framework segregates content delivery to the respective reconstructions, thereby enhancing the quality of the retrieved data. To our knowledge, we are the first to employ the GNN as a fundamental component to transform dense NLOS grid data into sparse structural features for efficient reconstruction. Comprehensive experiments demonstrate that our method attains the highest level of performance among existing methods across synthetic and real data. https://github.com/Nicholassu/DG-NLOS.

Problem

• Address high computational NLOS imaging demands
• Balance albedo and depth reconstruction challenges
• Enhance NLOS data quality via dual-branch framework

Innovation

• Dual-branch framework for segregation
• GNN transforms dense to sparse
• Albedo and depth focused reconstruction