ViiNeuS: Volumetric Initialization for Implicit Neural Surface reconstruction of urban scenes with limited image overlap

2024-03-15

AI Summary

Existing neural implicit surface reconstruction methods suffer from low accuracy and slow convergence in urban street-scene settings due to limited view overlap in monocular imagery, absence of LiDAR measurements, and lack of geometric priors. To address this, we propose an unsupervised, efficient neural implicit surface reconstruction framework. Our approach introduces two key innovations: (1) a novel hybrid implicit field architecture enabling progressive representation transfer from a volumetric density field to a signed distance function (SDF); and (2) a self-supervised, probability-density-guided volume rendering strategy that enables robust, prior-free SDF initialization. The method operates solely on sparse, overlapping monocular street-view images—without auxiliary sensors or explicit shape constraints. Evaluated on four large-scale outdoor driving datasets, our method achieves state-of-the-art reconstruction accuracy with fine-grained geometric detail and accelerates training by 2× compared to prior approaches.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Unsupervised & Self-Supervised Learning
• Search and Optimization: Learning to Search

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Neural implicit surface representation methods have recently shown impressive 3D reconstruction results. However, existing solutions struggle to reconstruct driving scenes due to their large size, highly complex nature and their limited visual observation overlap. Hence, to achieve accurate reconstructions, additional supervision data such as LiDAR, strong geometric priors, and long training times are required. To tackle such limitations, we present ViiNeuS, a new hybrid implicit surface learning method that efficiently initializes the signed distance field to reconstruct large driving scenes from 2D street view images. ViiNeuS's hybrid architecture models two separate implicit fields: one representing the volumetric density of the scene, and another one representing the signed distance to the surface. To accurately reconstruct urban outdoor driving scenarios, we introduce a novel volume-rendering strategy that relies on self-supervised probabilistic density estimation to sample points near the surface and transition progressively from volumetric to surface representation. Our solution permits a proper and fast initialization of the signed distance field without relying on any geometric prior on the scene, compared to concurrent methods. By conducting extensive experiments on four outdoor driving datasets, we show that ViiNeuS can learn an accurate and detailed 3D surface representation of various urban scene while being two times faster to train compared to previous state-of-the-art solutions.

Problem

• Neural Implicit Surfaces
• Large-Scale Urban Scene Reconstruction
• View Occlusion

Innovation

• ViiNeuS
• 3D Reconstruction
• Efficient Learning