Auto3R: Automated 3D Reconstruction and Scanning via Data-driven Uncertainty Quantification

2025-12-04

AI Summary

Addressing the challenge of fully automated, high-precision 3D scanning of objects with complex materials (e.g., specular or non-Lambertian surfaces) in real-world scenes, this paper introduces the first data-driven uncertainty quantification model for autonomous viewpoint selection under unknown geometry and appearance conditions. Our method integrates an iterative 3D reconstruction framework with a deep uncertainty prediction network to form a closed-loop active perception system deployable end-to-end on robotic platforms. Key contributions include: (1) the first incorporation of uncertainty quantification into 3D scanning path planning—eliminating manual intervention; (2) significantly improved robustness in reconstructing specular and highly reflective objects; and (3) empirical validation on a physical robotic arm–camera system, producing photorealistic, production-ready 3D digital assets. Experiments demonstrate that our approach consistently outperforms state-of-the-art methods in both reconstruction accuracy and scene coverage.

Technology Category

• Computer Vision: 3D Computer Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Security and Privacy: Data transparency and provenance

Abstract

Traditional high-quality 3D scanning and reconstruction typically relies on human labor to plan the scanning procedure. With the rapid development of embodied systems such as drones and robots, there is a growing demand of performing accurate 3D scanning and reconstruction in an fully automated manner. We introduce Auto3R, a data-driven uncertainty quantification model that is designed to automate the 3D scanning and reconstruction of scenes and objects, including objects with non-lambertian and specular materials. Specifically, in a process of iterative 3D reconstruction and scanning, Auto3R can make efficient and accurate prediction of uncertainty distribution over potential scanning viewpoints, without knowing the ground truth geometry and appearance. Through extensive experiments, Auto3R achieves superior performance that outperforms the state-of-the-art methods by a large margin. We also deploy Auto3R on a robot arm equipped with a camera and demonstrate that Auto3R can be used to effectively digitize real-world 3D objects and delivers ready-to-use and photorealistic digital assets. Our homepage: https://tomatoma00.github.io/auto3r.github.io .

Problem

• Automates 3D scanning and reconstruction using data-driven uncertainty quantification.
• Predicts uncertainty in scanning viewpoints without ground truth geometry.
• Digitizes real-world objects into photorealistic digital assets via robotic systems.

Innovation

• Data-driven uncertainty quantification for automated 3D scanning
• Iterative reconstruction with viewpoint uncertainty prediction
• Deployment on robot arm for real-world object digitization