CoStruction: Conjoint radiance field optimization for urban scene reconStruction with limited image overlap

2025-01-07

AI Summary

In urban driving scenarios, low inter-frame overlap and complex road topology severely degrade the accuracy and geometric completeness of 3D road surface reconstruction from vehicle-mounted video. Method: We propose a joint radiance field optimization framework featuring: (i) a novel cross-representation uncertainty estimation mechanism to filter ambiguous geometry; (ii) the first integration of radiance field joint optimization with uncertainty-guided hierarchical sampling, overcoming implicit surface reconstruction bottlenecks under sparse-view conditions; and (iii) hybrid implicit surface modeling to enhance fine-grained geometric fidelity. Results: Evaluated on mainstream autonomous driving datasets, our method significantly outperforms state-of-the-art approaches—particularly under large-scale, sparse-view settings—achieving substantial improvements in both geometric completeness and surface detail reconstruction accuracy.

Technology Category

• Computer Vision: 3D Computer Vision
• Reasoning under Uncertainty: Uncertainty Representations
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Reconstructing the surrounding surface geometry from recorded driving sequences poses a significant challenge due to the limited image overlap and complex topology of urban environments. SoTA neural implicit surface reconstruction methods often struggle in such setting, either failing due to small vision overlap or exhibiting suboptimal performance in accurately reconstructing both the surface and fine structures. To address these limitations, we introduce CoStruction, a novel hybrid implicit surface reconstruction method tailored for large driving sequences with limited camera overlap. CoStruction leverages cross-representation uncertainty estimation to filter out ambiguous geometry caused by limited observations. Our method performs joint optimization of both radiance fields in addition to guided sampling achieving accurate reconstruction of large areas along with fine structures in complex urban scenarios. Extensive evaluation on major driving datasets demonstrates the superiority of our approach in reconstructing large driving sequences with limited image overlap, outperforming concurrent SoTA methods.

Problem

• Image Mismatch
• Urban Environment
• Road Surface Reconstruction

Innovation

• Urban Scene Reconstruction
• Low Overlap Frames
• Optimized Brightness and Sampling