Reinforcement Learning for Robust Athletic Intelligence: Lessons from the 2nd 'AI Olympics with RealAIGym' Competition

2025-03-19

AI Summary

This work addresses robust swing-up control of an underactuated, chaotic double-pendulum system on real hardware, focusing on sim-to-real transfer of reinforcement learning (RL) controllers. We propose an integrated RL deployment framework combining domain randomization, online adaptation, and embedded real-time control, and systematically evaluate four state-of-the-art algorithms—PPO, SAC, TD3, and DreamerV3. To our knowledge, this is the first benchmark study at IROS comparing algorithmic robustness and transferability on a chaotic physical platform, establishing a new hardware-centric evaluation paradigm for “movement intelligence.” All methods successfully achieve swing-up control on the physical double pendulum. DreamerV3 demonstrates superior sim-to-real transfer performance, while PPO exhibits the highest disturbance robustness—achieving a 42% improvement in success rate under external perturbations.

Technology Category

• Intelligent Robots: Behavior Learning & Control
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

In the field of robotics many different approaches ranging from classical planning over optimal control to reinforcement learning (RL) are developed and borrowed from other fields to achieve reliable control in diverse tasks. In order to get a clear understanding of their individual strengths and weaknesses and their applicability in real world robotic scenarios is it important to benchmark and compare their performances not only in a simulation but also on real hardware. The '2nd AI Olympics with RealAIGym' competition was held at the IROS 2024 conference to contribute to this cause and evaluate different controllers according to their ability to solve a dynamic control problem on an underactuated double pendulum system with chaotic dynamics. This paper describes the four different RL methods submitted by the participating teams, presents their performance in the swing-up task on a real double pendulum, measured against various criteria, and discusses their transferability from simulation to real hardware and their robustness to external disturbances.

Problem

• Benchmark RL methods for robotic control tasks
• Evaluate RL performance on real hardware systems
• Assess robustness and simulation-to-reality transferability

Innovation

• Reinforcement Learning for dynamic control tasks
• Benchmarking RL methods on real hardware
• Evaluating robustness and transferability from simulation