Real-World Cooperative Bimanual Dexterous Grasp of Large Objects from Single-View Observations

2026-08-10

AI Summary

This work addresses the challenge of dexterous bimanual grasping of large objects in real-world settings, where existing approaches typically rely on complete 3D object models, are confined to simulation, or support only sequential manipulation. We propose the first end-to-end bimanual grasping framework capable of generating physically feasible, joint-level grasp configurations from a single-view observation. Our method leverages segmented point clouds and multimodal sensory inputs—including vision, joint angles, and force signals—to drive a denoising diffusion probabilistic model (DDPM), integrated with motion planning and online grasp optimization for stable execution. Evaluated on a real dual-arm robot, the system achieves high success rates across diverse large objects with unknown geometry and poses. Ablation studies confirm the contribution of each component, demonstrating, for the first time, model-free cooperative bimanual dexterous grasping in real environments without requiring full 3D object reconstructions.

Abstract

Bimanual dexterous grasping of large objects is a critical challenge in robotic manipulation. However, most existing studies focus on sequential manipulation rather than cooperative grasping, and methods addressing such bimanual tasks have largely been limited to simulation. These limitations stem from the difficulty of acquiring full 3D object models and generating physically plausible grasping actions. To fill this gap, we propose a real-world bimanual grasping framework that includes: a multimodal dataset capturing joint angles, visual observations and force signals; a Denoising Diffusion Probabilistic Model (DDPM)-based module that generates joint-level grasp configurations from segmented point clouds; and an execution strategy that integrates motion planning with online grasp refinement to ensure physical stability and feasibility. Our approach enables the synthesis of executable bimanual grasps from single-view inputs, reducing dependence on complete 3D object models and ensuring stable real-world performance. Experiments on a dual-arm robot demonstrate high success rates across unseen objects with varying geometries and poses, and ablation studies confirm the contributions of key components of our system.

Problem

• bimanual grasping
• dexterous manipulation
• large objects
• single-view observation
• real-world robotics

Innovation

• bimanual grasping
• denoising diffusion probabilistic model
• single-view observation
• real-world robotic manipulation
• multimodal dataset