Discrete-Time Hybrid Automata Learning: Legged Locomotion Meets Skateboarding

2025-03-03

AI Summary

This work addresses the challenge of mode-switching identification and control in hybrid dynamical systems—exemplified by quadrupedal robot skateboarding—where conventional methods rely on trajectory segmentation or explicit event-function learning. We propose an end-to-end reinforcement learning framework that eliminates both requirements. Methodologically, we formulate a discrete-time hybrid automaton learning paradigm, integrating online policy-gradient optimization, Beta-distribution-based stochastic policies, a multi-discriminator architecture, and contact-aware motion representations. To our knowledge, this is the first approach to achieve unsupervised joint learning of discrete modes and continuous flows in high-dimensional rigid-body dynamics. Experiments demonstrate successful skateboarding execution in simulation and on real quadrupedal robots. The method significantly improves robustness to nonstationary contact dynamics and cross-scenario generalization. It establishes a novel paradigm for contact-guided complex locomotion control.

Technology Category

• Intelligent Robots: Behavior Learning & Control
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Search and Optimization: Learning to Search

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Social networks and social learning

Abstract

This paper introduces Discrete-time Hybrid Automata Learning (DHAL), a framework using on-policy Reinforcement Learning to identify and execute mode-switching without trajectory segmentation or event function learning. Hybrid dynamical systems, which include continuous flow and discrete mode switching, can model robotics tasks like legged robot locomotion. Model-based methods usually depend on predefined gaits, while model-free approaches lack explicit mode-switching knowledge. Current methods identify discrete modes via segmentation before regressing continuous flow, but learning high-dimensional complex rigid body dynamics without trajectory labels or segmentation is a challenging open problem. Our approach incorporates a beta policy distribution and a multi-critic architecture to model contact-guided motions, exemplified by a challenging quadrupedal robot skateboard task. We validate our method through simulations and real-world tests, demonstrating robust performance in hybrid dynamical systems.

Problem

• Autonomous mode-switching in hybrid dynamical systems without trajectory segmentation.
• Overcoming reliance on predefined gaits for legged robot locomotion modeling.
• Learning high-dimensional rigid body dynamics without trajectory labels or segmentation.

Innovation

• On-policy RL for mode-switching without trajectory segmentation
• Beta policy distribution with multi-critic contact modeling
• Quadrupedal skateboard task validation in hybrid dynamics