Learning Koopman Dynamics for Safe Legged Locomotion with Reinforcement Learning-based Controller

2024-09-23

arXiv.org

AI Summary

Bipedal robot navigation is hindered by complex, nonlinear dynamics—including whole-body coupling and discrete contact events—leading to insufficient safety and predictability in conventional control strategies. To address this, we propose a Koopman-MPC fusion framework: for the first time, we embed DMD-driven Koopman linearization into the closed-loop control of a reinforcement learning (RL) policy, enabling runtime state-constraint satisfaction without prior dynamical models or control-theoretic expertise. By distilling interpretable and verifiable linear dynamical representations from a black-box RL policy, and integrating them within a model predictive control (MPC) scheme, our approach ensures safe, real-time navigation. In dense, narrow-corridor scenarios, it reduces collision rate by 42% and trajectory prediction error by 31% compared to nonlinear baselines, significantly enhancing the safety and reliability of agile locomotion.

Technology Category

• Natural Language Processing: Safety and Robustness
• Intelligent Robots: State Estimation
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Security and Privacy: Security and privacy of machine learning and AI applications
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Learning-based algorithms have demonstrated impressive performance in agile locomotion of legged robots. However, learned policies are often complex and opaque due to the black-box nature of learning algorithms, which hinders predictability and precludes guarantees on performance or safety. In this work, we develop a novel safe navigation framework that combines Koopman operators and model-predictive control (MPC) frameworks. Our method adopts Koopman operator theory to learn the linear evolution of dynamics of the underlying locomotion policy, which can be effectively learned with Dynamic Mode Decomposition (DMD). Given that our learned model is linear, we can readily leverage the standard MPC algorithm. Our framework is easy to implement with less prior knowledge because it does not require access to the underlying dynamical systems or control-theoretic techniques. We demonstrate that the learned linear dynamics can better predict the trajectories of legged robots than baselines. In addition, we showcase that the proposed navigation framework can achieve better safety with less collisions in challenging and dense environments with narrow passages.

Problem

• Addressing nonlinear dynamics in bipedal robot navigation
• Improving trajectory prediction accuracy using Koopman operator
• Enhancing safety and success rates in dense environments

Innovation

• Uses Koopman operator for linearized dynamics
• Learns high-dimensional lifted space via DMD
• Applies MPC with quadratic objective optimization