On learning racing policies with reinforcement learning

2025-04-03

AI Summary

To address the insufficient reliability of fully autonomous driving under extreme operating conditions, this paper proposes an end-to-end reinforcement learning (RL) framework designed for zero-shot deployment on real vehicles, using autonomous racing as a representative task. Methodologically, it integrates domain randomization, high-fidelity actuator dynamics modeling, and a lightweight CNN-LSTM policy network, trained via PPO or SAC algorithms to yield high-performance driving policies. The key contributions are threefold: (1) the first demonstration of an RL policy achieving zero-shot deployment on the F1TENTH platform that outperforms human expert drivers; (2) superior performance over state-of-the-art model predictive control (MPC) approaches in extreme dynamic maneuvers; and (3) empirical validation of deep RL’s feasibility and advantages for极限 dynamic vehicle control, establishing a transferable technical pathway toward highly reliable autonomous driving.

Technology Category

• Machine Learning: Reinforcement Learning
• Planning, Routing, and Scheduling: Learning for Planning and Scheduling
• Search and Optimization: Learning to Search

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Uses of LLMs and GenAI for marketplace design, bidding, and strategic interactions

Abstract

Fully autonomous vehicles promise enhanced safety and efficiency. However, ensuring reliable operation in challenging corner cases requires control algorithms capable of performing at the vehicle limits. We address this requirement by considering the task of autonomous racing and propose solving it by learning a racing policy using Reinforcement Learning (RL). Our approach leverages domain randomization, actuator dynamics modeling, and policy architecture design to enable reliable and safe zero-shot deployment on a real platform. Evaluated on the F1TENTH race car, our RL policy not only surpasses a state-of-the-art Model Predictive Control (MPC), but, to the best of our knowledge, also represents the first instance of an RL policy outperforming expert human drivers in RC racing. This work identifies the key factors driving this performance improvement, providing critical insights for the design of robust RL-based control strategies for autonomous vehicles.

Problem

• Develops RL policy for autonomous racing vehicles
• Enhances performance beyond MPC and human drivers
• Ensures safe zero-shot deployment on real platforms

Innovation

• Uses Reinforcement Learning for racing policy
• Leverages domain randomization and actuator modeling
• First RL policy outperforming human drivers