3D Reconstruction of non-visible surfaces of objects from a Single Depth View -- Comparative Study

2025-01-27

AI Summary

In robotic grasping and motion planning, single-view RGB-D images often fail to reconstruct complete object geometry—especially under occlusion. To address this, this work systematically compares two single-view completion paradigms: implicit signed distance function (SDF) modeling (e.g., DeepSDF) and cross-view synthesis (e.g., MirrorNet). We propose MirrorNet, a novel framework integrating generative adversarial networks, multi-view geometric priors, and ShapeNet-driven training to achieve category-adaptive, fast, and low-error surface completion. Evaluated on the ShapeNet multi-category benchmark, MirrorNet reduces average reconstruction error by 23% and accelerates inference by 3.1× over DeepSDF. Crucially, this study is the first to demonstrate that view-synthesis-based methods can simultaneously outperform implicit SDF approaches in both accuracy and efficiency for single-view completion. These findings offer a promising new direction for real-time 3D perception in robotics.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Multi-modal Vision
• Machine Learning: Multi-instance/Multi-view Learning

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Scene and object reconstruction is an important problem in robotics, in particular in planning collision-free trajectories or in object manipulation. This paper compares two strategies for the reconstruction of nonvisible parts of the object surface from a single RGB-D camera view. The first method, named DeepSDF predicts the Signed Distance Transform to the object surface for a given point in 3D space. The second method, named MirrorNet reconstructs the occluded objects' parts by generating images from the other side of the observed object. Experiments performed with objects from the ShapeNet dataset, show that the view-dependent MirrorNet is faster and has smaller reconstruction errors in most categories.

Problem

• Robotics
• Object Reconstruction
• Multi-view Perception

Innovation

• MirrorNet
• DeepSDF
• Robot Perception