🤖 AI Summary
To address the challenge of view selection in active neural 3D reconstruction under sparse and imbalanced viewpoint distributions, this paper proposes a novel uncertainty modeling and view selection method based on Colored Surface Voxels (CSV). Unlike conventional approaches that separately model geometry and appearance or assess visibility only after convergence, our method jointly encodes surface geometry and color uncertainty at the voxel level. It further employs surface-guided 3D spatial aggregation to accurately identify uncertainty in occluded and geometrically complex regions. We integrate this uncertainty representation into a NeRF-driven active view selection framework. Extensive experiments on DTU, Blender, and a custom-imbalanced viewpoint dataset demonstrate that our method achieves up to 30% improvement in reconstruction accuracy, significantly outperforming state-of-the-art methods.
📝 Abstract
Active view selection in 3D scene reconstruction has been widely studied since training on informative views is critical for reconstruction. Recently, Neural Radiance Fields (NeRF) variants have shown promising results in active 3D reconstruction using uncertainty-guided view selection. They utilize uncertainties estimated with neural networks that encode scene geometry and appearance. However, the choice of uncertainty integration methods, either voxel-based or neural rendering, has conventionally depended on the types of scene uncertainty being estimated, whether geometric or appearance-related. In this paper, we introduce Colorized Surface Voxel (CSV)-based view selection, a new next-best view (NBV) selection method exploiting surface voxel-based measurement of uncertainty in scene appearance. CSV encapsulates the uncertainty of estimated scene appearance (e.g., color uncertainty) and estimated geometric information (e.g., surface). Using the geometry information, we interpret the uncertainty of scene appearance 3D-wise during the aggregation of the per-voxel uncertainty. Consequently, the uncertainty from occluded and complex regions is recognized under challenging scenarios with limited input data. Our method outperforms previous works on popular datasets, DTU and Blender, and our new dataset with imbalanced viewpoints, showing that the CSV-based view selection significantly improves performance by up to 30%.