Is Image-based Object Pose Estimation Ready to Support Grasping?

2025-12-01

AI Summary

This work investigates whether single-RGB-image-based 6D instance-level object pose estimation can serve as the sole perceptual input for robotic grasping. To this end, we establish the first systematic evaluation framework tailored to physics-based simulated grasping tasks, integrating both parallel-jaw grippers and underactuated robotic hands within Gazebo. Grasps are executed in closed-loop 3D using estimated poses, and five state-of-the-art open-source pose estimators are benchmarked on a subset of the BOP dataset. Unlike conventional offline pose accuracy metrics, our approach directly links pose estimation performance to task-level success—grasp success rate. Experimental results show that several purely vision-based methods achieve over 70% stable grasp success in simulation, demonstrating their viability as lightweight, cost-effective perception solutions for grasping. This study bridges a critical gap between visual pose estimation and end-to-end robotic manipulation evaluation, providing a reproducible benchmark and principled algorithm selection guidance for vision-driven grasping systems.

Technology Category

• Intelligent Robots: State Estimation
• Computer Vision: Biometrics, Face, Gesture & Pose
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization

Abstract

We present a framework for evaluating 6-DoF instance-level object pose estimators, focusing on those that require a single RGB (not RGB-D) image as input. Besides gaining intuition about how accurate these estimators are, we are interested in the degree to which they can serve as the sole perception mechanism for robotic grasping. To assess this, we perform grasping trials in a physics-based simulator, using image-based pose estimates to guide a parallel gripper and an underactuated robotic hand in picking up 3D models of objects. Our experiments on a subset of the BOP (Benchmark for 6D Object Pose Estimation) dataset compare five open-source object pose estimators and provide insights that were missing from the literature.

Problem

• Evaluating 6-DoF object pose estimation from single RGB images
• Assessing pose estimators for robotic grasping in simulation
• Comparing five open-source estimators on the BOP dataset

Innovation

• Evaluates 6-DoF pose estimators using single RGB images
• Assesses pose estimation for robotic grasping in simulation
• Compares five open-source estimators on BOP dataset subset