Learning to Race in Minutes: Infoprop Dyna on the Mini Wheelbot

2026-05-01

AI Summary

This work addresses the challenge of enabling robots with fast, nonlinear, and unstable dynamics to efficiently learn complex control tasks without relying on high-fidelity simulation or domain randomization. The authors propose Infoprop Dyna, an uncertainty-aware model-based reinforcement learning framework, which enables the underactuated single-wheel robot Mini Wheelbot to learn a high-speed lap-completion policy directly through real-world interaction. Requiring only 11 minutes of real-world experience, the method achieves stable high-velocity circular navigation, demonstrating exceptional sample efficiency. Notably, this is the first demonstration of successful sim-to-real-free transfer for a highly dynamic task without any simulation pre-training, highlighting the framework’s effectiveness and breakthrough potential for online learning on physical robots.

Abstract

Reinforcement Learning (RL) has the potential to enable robots with fast, nonlinear, and unstable dynamics to reach the limits of their performance. However, most recent advances rely on carefully designed physics-based simulators and domain randomization to achieve successful sim-to-real transfer within reasonable wall-clock time. In this work, we bypass the need for such simulators and demonstrate that Infoprop Dyna, a state-of-the-art uncertainty-aware model-based reinforcement learning (MBRL) framework, can enable robots to learn directly from real-world interactions. Using Infoprop Dyna, the Mini Wheelbot, an underactuated unicycle robot, learns to race around a track within 11 minutes of real-world experience.

Problem

• reinforcement learning
• sim-to-real transfer
• model-based reinforcement learning
• robot learning
• real-world interaction

Innovation

• model-based reinforcement learning
• uncertainty-aware
• sim-to-real transfer
• real-world learning
• Infoprop Dyna