Spatiotemporal Agility: Time-Constrained Reinforcement Learning for Vision-Guided Dynamic Quadrupedal Interception

2026-08-07

AI Summary

This work addresses the challenge of enabling quadrupedal robots to accurately intercept fast-moving dynamic targets under stringent time constraints, a task hindered by insufficient spatiotemporal precision and perception-to-control latency in existing velocity-tracking approaches. The authors propose an end-to-end dynamic interception framework that directly conditions the reinforcement learning policy on visually predicted target landing position and time of arrival, bypassing intermediate velocity commands to avoid error accumulation. Integrating multi-camera perception, online trajectory prediction, low-latency communication, and sim-to-real locomotion control, the system achieves the first closed-loop dynamic interception on a quadruped. Evaluated on toss-interception tasks within 2 meters and flight durations of 0.8–1.2 seconds, the method significantly outperforms velocity-tracking baselines in success rate and exhibits less performance degradation after real-world deployment, demonstrating its efficacy and robustness.

Abstract

Legged robots require robust agility to perceive and interact with complex and dynamic environments within a constrained time. However, most existing quadruped locomotion works rely on velocity-tracking policy, which struggle to reach precise targets within strict temporal constraints. Moreover, integrating real-time perception with agile locomotion for highly dynamic targets remains challenging due to sensor latency and processing delays. To concretely study and benchmark such agility in dynamic settings, we introduce a challenging ball-catching task for legged robots. This paper proposes an integrated framework that combines a vision module for landing point and time prediction with a direct position and time conditioned RL locomotion policy, instead of intermediate velocity commands. Beyond the method design, this work presents a system-level contribution that completes real-time robotic interception system that integrates multi-camera perception, online trajectory prediction, low-latency target communication, and sim-to-real locomotion control into a closed-loop deployment pipeline. By explicitly predicting the future spatial-temporal target, our approach mitigates perception latency during dynamic interception. We conducted extensive ball-catching experiments for the legged robot. Through comparative experiments against a velocity-tracking baseline, our direct target-conditioned approach achieves a higher success rate in catching balls with predicted landing spots within 2 meters and flight times between 0.8 and 1.2 seconds. This shows that the robot has successfully completed the dynamic ball-catching task under our tested setup. Furthermore, our policy exhibits a smaller performance gap after deployment, suggesting improved sim-to-real behavior in these trials.

Problem

• spatiotemporal agility
• dynamic interception
• time-constrained locomotion
• vision-guided control
• legged robots

Innovation

• spatiotemporal agility
• time-constrained reinforcement learning
• vision-guided interception
• direct target-conditioned policy
• sim-to-real locomotion