Neural shape reconstruction from multiple views with static pattern projection

2025-06-02

AI Summary

This work addresses high-precision 3D reconstruction under unknown and arbitrary relative motion between a camera and a projector. We propose a calibration-free dynamic active stereo reconstruction method that jointly optimizes camera/projector poses and object geometry within an end-to-end differentiable framework, integrating neural implicit signed distance field (SDF) modeling with voxel-based differentiable rendering. To enhance cross-view consistency, we incorporate static structured-light encoding and multi-view geometric constraints. To our knowledge, this is the first approach to synergistically combine neural SDFs and voxel differentiable rendering for self-calibrating active stereo reconstruction under motion—eliminating reliance on rigid hardware configurations or pre-calibration. Extensive evaluation on synthetic and real-world data demonstrates sub-millimeter reconstruction accuracy and significantly improved pose estimation over calibrated baselines, enabling markerless, flexible scanning.

Technology Category

• Computer Vision: 3D Computer Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Active-stereo-based 3D shape measurement is crucial for various purposes, such as industrial inspection, reverse engineering, and medical systems, due to its strong ability to accurately acquire the shape of textureless objects. Active stereo systems typically consist of a camera and a pattern projector, tightly fixed to each other, and precise calibration between a camera and a projector is required, which in turn decreases the usability of the system. If a camera and a projector can be freely moved during shape scanning process, it will drastically increase the convenience of the usability of the system. To realize it, we propose a technique to recover the shape of the target object by capturing multiple images while both the camera and the projector are in motion, and their relative poses are auto-calibrated by our neural signed-distance-field (NeuralSDF) using novel volumetric differential rendering technique. In the experiment, the proposed method is evaluated by performing 3D reconstruction using both synthetic and real images.

Problem

• Enables 3D shape reconstruction with freely moving camera and projector
• Auto-calibrates relative poses using NeuralSDF and volumetric rendering
• Improves usability for textureless object scanning in active stereo systems

Innovation

• NeuralSDF for auto-calibration of moving devices
• Volumetric differential rendering for shape recovery
• Multi-view capture with dynamic camera and projector