Beyond Visual Grasping: Benchmarking Complex Grasping from Detection to Execution

2026-07-15

AI Summary

Current grasping benchmarks predominantly focus on single-step visual perception, rendering them inadequate for evaluating complex grasping tasks that require scene-level reasoning and semantic understanding. To address this gap, this work proposes GCA-Bench—the first comprehensive benchmark tailored for complex grasping actions—incorporating semantic constraints and multi-step reasoning scenarios. The benchmark introduces novel evaluation metrics to enable end-to-end assessment under a unified experimental setting. By integrating conventional grasping pipelines with large-scale foundation models, GCA-Bench reveals that existing methods achieve success rates below 70% in complex scenarios. A detailed failure mode analysis further provides critical insights and directions for enhancing the robustness and generalization capabilities of grasping strategies.

Abstract

Robust robotic grasping remains a fundamental challenge for complex real-world applications. Recent advances in large-scale models demonstrate promising capabilities for reasoning in robotic tasks. However, existing benchmarks for grasping primarily focus on isolated, visual-based grasp pose detection, failing to capture the complexity of grasping tasks that require multi-step reasoning and semantic understanding during execution. To address this gap, we propose GCA-Bench, a benchmark featuring challenging \textit{grasping with complex action} scenarios that involve both scene-level reasoning and semantic constraints. GCA-Bench enables the evaluation of recent large foundation models under the same settings. To demonstrate the effectiveness of our new benchmark, we implement a diverse set of baselines, ranging from traditional grasp detection pipelines to end-to-end learning methods. Empirical studies achieve success rates below 70\% on complex grasping scenarios, underscoring critical limitations. In addition, we propose new evaluation metrics, analyze critical failure models, and provide insights to guide the development of more robust and generalizable grasping strategies.

Problem

• robotic grasping
• complex action
• semantic understanding
• multi-step reasoning
• benchmarking

Innovation

• complex grasping
• foundation models
• semantic reasoning
• robotic benchmarking
• multi-step execution