RLPP: A Residual Method for Zero-Shot Real-World Autonomous Racing on Scaled Platforms

2025-01-28

AI Summary

Autonomous racing on unknown tracks demands zero-shot, real-time control without prior track knowledge or real-world fine-tuning—posing significant Sim-to-Real transfer and robustness challenges. Method: This paper proposes RLPP, a residual reinforcement learning framework that tightly couples a kinematically consistent Pure Pursuit (PP) controller—as an interpretable, model-based prior—with an end-to-end residual policy trained via Proximal Policy Optimization (PPO). The two components are jointly optimized in simulation for closed-loop trajectory tracking under accurate vehicle kinematics modeling. Contribution/Results: RLPP establishes the first deep synergy between PP and RL, markedly mitigating Sim-to-Real mismatch. Deployed directly on the F1TENTH embedded platform without real-data adaptation, it achieves 6.37% lap-time reduction, an 8× reduction in Sim-to-Real performance degradation, and attains over 52% of state-of-the-art (SOTA) performance under zero-shot deployment—establishing a new paradigm for lightweight, interpretable, and highly robust real-world racing control.

Technology Category

• Planning, Routing, and Scheduling: Learning for Planning and Scheduling
• Search and Optimization: Learning to Search
• Intelligent Robots: Learning & Optimization for ROB

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization

Abstract

Autonomous racing presents a complex environment requiring robust controllers capable of making rapid decisions under dynamic conditions. While traditional controllers based on tire models are reliable, they often demand extensive tuning or system identification. RL methods offer significant potential due to their ability to learn directly from interaction, yet they typically suffer from the Sim-to-Reall gap, where policies trained in simulation fail to perform effectively in the real world. In this paper, we propose RLPP, a residual RL framework that enhances a PP controller with an RL-based residual. This hybrid approach leverages the reliability and interpretability of PP while using RL to fine-tune the controller's performance in real-world scenarios. Extensive testing on the F1TENTH platform demonstrates that RLPP improves lap times by up to 6.37 %, closing the gap to the SotA methods by more than 52 % and providing reliable performance in zero-shot real-world deployment, overcoming key challenges associated with the Sim-to-Real transfer and reducing the performance gap from simulation to reality by more than 8-fold when compared to the baseline RL controller. The RLPP framework is made available as an open-source tool, encouraging further exploration and advancement in autonomous racing research. The code is available at: www.github.com/forzaeth/rlpp.

Problem

• Autonomous Racing
• Rapid Decision-making
• Adaptive Learning

Innovation

• Reinforcement Learning
• PP Controller Optimization
• Sim-to-Real Adaptation