Learning coordinated badminton skills for legged manipulators.

2025-05-28

Science Robotics

AI Summary

Legged mobile manipulators face significant challenges in motion-perception co-adaptation under dynamic adversarial scenarios. Method: This work presents the first end-to-end whole-body sensorimotor reinforcement learning framework enabling autonomous badminton play. It integrates real-time visual tracking, quadrupedal dynamic gait planning, and precise 7-DOF manipulator stroke control. To bridge the sim-to-real gap, we introduce perception noise modeling calibrated on real camera data. Additionally, a badminton trajectory prediction model and constraint-aware policy optimization enhance closed-loop robustness. Results: Experiments demonstrate that the system reliably predicts opponent shots, autonomously navigates within the service court, and executes high-accuracy returns across diverse scenarios. This is the first empirical validation of legged mobile manipulators performing high-speed, physically interactive sports competitions—establishing feasibility for real-world adversarial robotic athletics.

Technology Category

• Intelligent Robots: Manipulation
• Multiagent Systems: Adversarial Agents
• Machine Learning: Adversarial Learning & Robustness

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Coordinating the motion between lower and upper limbs and aligning limb control with perception are substantial challenges in robotics, particularly in dynamic environments. To this end, we introduce an approach for enabling legged mobile manipulators to play badminton, a task that requires precise coordination of perception, locomotion, and arm swinging. We propose a unified reinforcement learning-based control policy for whole-body visuomotor skills involving all degrees of freedom to achieve effective shuttlecock tracking and striking. This policy is informed by a perception noise model that uses real-world camera data, allowing for consistent perception error levels between simulation and deployment and encouraging learned active perception behaviors. Our method includes a shuttlecock prediction model and constrained reinforcement learning for robust motion control to enhance deployment readiness. Extensive experimental results in a variety of environments validate the robot's capability to predict shuttlecock trajectories, navigate the service area effectively, and execute precise strikes against human players, demonstrating the feasibility of using legged mobile manipulators in complex and dynamic sports scenarios.

Problem

• Coordinating limb motion and perception in dynamic environments
• Developing whole-body control for badminton-playing legged manipulators
• Ensuring robust shuttlecock tracking and striking accuracy

Innovation

• Unified reinforcement learning for whole-body control
• Perception noise model using real-world camera data
• Shuttlecock prediction and constrained reinforcement learning