RoboDuet: Learning a Cooperative Policy for Whole-body Legged Loco-Manipulation

2024-03-26

AI Summary

Coordinating whole-body locomotion and manipulation for quadrupedal robots equipped with robotic arms remains challenging due to tight coupling between legged locomotion and arm-end-effector dynamics. Method: This paper proposes RoboDuet, a dual-strategy synergistic framework enabling real-time locomotion-manipulation coupling. It introduces a novel decoupled–coupled architecture, integrating reinforcement learning training, full-state feedback modeling, hardware-in-the-loop optimization on real platforms, and cross-platform policy generalization—enabling zero-shot transfer across robots. Contribution/Results: RoboDuet overcomes limitations in 6-DoF end-effector pose tracking range and whole-body dynamic coordination inherent in conventional single-strategy approaches. In complex locomotion-manipulation (locomanip) tasks, it achieves a 23% improvement in success rate. The framework has been successfully deployed in real time and validated for generalization across multiple isomorphic quadrupedal robot platforms.

Technology Category

• Intelligent Robots: Manipulation
• Humans and AI: Planning and Decision Support for Human-Machine Teams
• Planning, Routing, and Scheduling: Learning for Planning and Scheduling

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Fully leveraging the loco-manipulation capabilities of a quadruped robot equipped with a robotic arm is non-trivial, as it requires controlling all degrees of freedom (DoFs) of the quadruped robot to achieve effective whole-body coordination. In this letter, we propose a novel framework RoboDuet, which employs two collaborative policies to realize locomotion and manipulation simultaneously, achieving whole-body control through mutual interactions. Beyond enabling large-range 6D pose tracking for manipulation, we find that the two-policy framework supports zero-shot transfer across quadruped robots with similar morphology and physical dimensions in the real world. Our experiments demonstrate that RoboDuet achieves a 23% improvement in success rate over the baseline in challenging loco-manipulation tasks employing whole-body control. To support further research, we provide open-source code and additional videos on our website: locomanip-duet.github.io.

Problem

• Achieving whole-body coordination in quadruped robots
• Simultaneous locomotion and manipulation control
• Zero-shot transfer across similar morphology robots

Innovation

• Collaborative policies for locomotion
• Whole-body coordination through mutual interactions
• Zero-shot transfer across similar robots