GURecon: Learning Detailed 3D Geometric Uncertainties for Neural Surface Reconstruction

2024-12-19

arXiv.org

AI Summary

In neural surface reconstruction, the absence of ground-truth mesh supervision hinders reliable geometric quality assessment. To address this, we propose a geometrically consistent framework for learning continuous 3D uncertainty fields. Our method introduces an online knowledge distillation mechanism that enables uncertainty estimation without access to true geometric labels. We further design a lighting-decoupled implicit field to mitigate photometric optimization bias on geometry estimation, thereby improving robustness. Additionally, our architecture is NeRF-compatible, supporting plug-and-play integration with diverse neural surface representations. Extensive experiments across multiple benchmarks demonstrate significant improvements in uncertainty modeling fidelity. Moreover, the learned uncertainty fields substantially enhance downstream tasks—particularly incremental surface reconstruction—by enabling more reliable geometric reasoning and data selection. The framework achieves state-of-the-art performance in uncertainty-aware reconstruction while maintaining computational efficiency and broad compatibility.

Technology Category

• Reasoning under Uncertainty: Uncertainty Representations
• Machine Learning: Calibration & Uncertainty Quantification
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Neural surface representation has demonstrated remarkable success in the areas of novel view synthesis and 3D reconstruction. However, assessing the geometric quality of 3D reconstructions in the absence of ground truth mesh remains a significant challenge, due to its rendering-based optimization process and entangled learning of appearance and geometry with photometric losses. In this paper, we present a novel framework, i.e, GURecon, which establishes a geometric uncertainty field for the neural surface based on geometric consistency. Different from existing methods that rely on rendering-based measurement, GURecon models a continuous 3D uncertainty field for the reconstructed surface, and is learned by an online distillation approach without introducing real geometric information for supervision. Moreover, in order to mitigate the interference of illumination on geometric consistency, a decoupled field is learned and exploited to finetune the uncertainty field. Experiments on various datasets demonstrate the superiority of GURecon in modeling 3D geometric uncertainty, as well as its plug-and-play extension to various neural surface representations and improvement on downstream tasks such as incremental reconstruction. The code and supplementary material are available on the project website: https://zju3dv.github.io/GURecon/.

Problem

• Assessing geometric quality without ground truth mesh
• Modeling continuous 3D uncertainty for neural surfaces
• Mitigating illumination interference on geometric consistency

Innovation

• Establishes geometric uncertainty field via consistency
• Uses online distillation without geometric supervision
• Learns decoupled field to reduce illumination interference