Seeing A 3D World in A Grain of Sand

2025-03-01

AI Summary

Omnidirectional 3D reconstruction of millimeter-scale miniature scenes from a single snapshot remains challenging due to extreme viewpoint sparsity and severe ill-posedness. Method: We propose a cat’s-eye imaging system integrating eight custom-designed nested pyramidal mirror arrays with a single camera to enable synchronous, full环绕 multi-view acquisition. We introduce visual hull depth priors into the 3D Gaussian Splatting (3DGS) framework and jointly optimize via multi-view geometry constraints to mitigate reconstruction ambiguity under ultra-sparse views. Contribution/Results: Our key innovations include the nested pyramidal mirror layout enabling wide-coverage snapshot imaging, and visual hull–guided end-to-end 3DGS optimization. Evaluated on both synthetic and real miniature scenes (e.g., dollhouse scale), our method achieves state-of-the-art performance, enabling high-fidelity novel-view synthesis and sub-millimeter-accurate digital twin reconstruction.

Technology Category

• Computer Vision: 3D Computer Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Security and Privacy: Large-scale security measurements
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

We present a snapshot imaging technique for recovering 3D surrounding views of miniature scenes. Due to their intricacy, miniature scenes with objects sized in millimeters are difficult to reconstruct, yet miniatures are common in life and their 3D digitalization is desirable. We design a catadioptric imaging system with a single camera and eight pairs of planar mirrors for snapshot 3D reconstruction from a dollhouse perspective. We place paired mirrors on nested pyramid surfaces for capturing surrounding multi-view images in a single shot. Our mirror design is customizable based on the size of the scene for optimized view coverage. We use the 3D Gaussian Splatting (3DGS) representation for scene reconstruction and novel view synthesis. We overcome the challenge posed by our sparse view input by integrating visual hull-derived depth constraint. Our method demonstrates state-of-the-art performance on a variety of synthetic and real miniature scenes.

Problem

• Recovering 3D views of miniature scenes
• Overcoming challenges in sparse view input
• Customizable mirror design for optimized view coverage

Innovation

• Catadioptric system with single camera and mirrors
• Customizable nested pyramid mirror design
• 3D Gaussian Splatting for scene reconstruction