Benchmarking the Effects of Object Pose Estimation and Reconstruction on Robotic Grasping Success

2026-02-19

AI Summary

This study addresses the lack of quantitative evaluation linking 3D reconstruction quality to downstream robotic grasping performance. The authors establish a large-scale physics-based simulation benchmark that, for the first time, maps the joint errors of 3D reconstruction and 6D pose estimation onto grasp success rates. Through an integrated pipeline encompassing multi-view reconstruction, pose estimation, mesh geometry analysis, and physical grasping simulation, they demonstrate that while reconstruction artifacts substantially reduce the number of feasible grasp candidates, their impact on final success is limited when pose estimates are accurate. Crucially, grasp performance is predominantly governed by pose estimation errors—particularly translational errors, which alone serve as a strong predictor of grasping outcomes, especially for symmetric objects.

Technology Category

• Intelligent Robots: State Estimation
• Computer Vision: Biometrics, Face, Gesture & Pose
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

3D reconstruction serves as the foundational layer for numerous robotic perception tasks, including 6D object pose estimation and grasp pose generation. Modern 3D reconstruction methods for objects can produce visually and geometrically impressive meshes from multi-view images, yet standard geometric evaluations do not reflect how reconstruction quality influences downstream tasks such as robotic manipulation performance. This paper addresses this gap by introducing a large-scale, physics-based benchmark that evaluates 6D pose estimators and 3D mesh models based on their functional efficacy in grasping. We analyze the impact of model fidelity by generating grasps on various reconstructed 3D meshes and executing them on the ground-truth model, simulating how grasp poses generated with an imperfect model affect interaction with the real object. This assesses the combined impact of pose error, grasp robustness, and geometric inaccuracies from 3D reconstruction. Our results show that reconstruction artifacts significantly decrease the number of grasp pose candidates but have a negligible effect on grasping performance given an accurately estimated pose. Our results also reveal that the relationship between grasp success and pose error is dominated by spatial error, and even a simple translation error provides insight into the success of the grasping pose of symmetric objects. This work provides insight into how perception systems relate to object manipulation using robots.

Problem

• 3D reconstruction
• 6D object pose estimation
• robotic grasping
• grasp success
• pose error

Innovation

• 3D reconstruction
• 6D pose estimation
• robotic grasping
• physics-based benchmark
• grasp robustness