PlanRL: A Trajectory Planning Architecture for Reinforcement Learning-based Driving Experts

2026-06-25

AI Summary

This work addresses key limitations of existing reinforcement learning (RL) approaches for autonomous driving—namely, poor interpretability, difficulty in modeling road geometry, and weak compatibility with modern planning architectures—by proposing a novel RL framework that integrates the Frenet coordinate system with polynomial trajectory planning. The method introduces a kinematic feasibility check after policy inference to generate trajectories that respect vehicle dynamic constraints. To the best of our knowledge, this is the first approach to embed Frenet coordinates and analytical trajectory planning within an end-to-end RL pipeline, substantially enhancing interpretability and reducing learning complexity. Evaluated on the CARLA Offline Leaderboard v1 and NoCrash benchmarks, the proposed method achieves a 5% and 11% improvement in driving score, and an 8% and 19% increase in task success rate, respectively, significantly outperforming current control-oriented RL baselines.

Abstract

Reinforcement learning (RL) has become a prominent framework for developing driving experts in autonomous vehicles. However, most existing RL-based experts are designed to output direct control commands (e.g., throttle, steering), which suffer from a lack of interpretability, high spatial complexity in learning road geometries, and poor compatibility with modern end-to-end planning architectures. To address these limitations, we propose a novel trajectory planning architecture for RL driving experts that integrates an RL policy with a polynomial-based trajectory planner. By employing a Frenet-frame coordinate system, our method simplifies complex road geometries into a curvilinear framework, offering a structured coordinate prior that facilitates policy learning. Furthermore, we incorporate a kinematic feasibility check into the planning stage to ensure that generated trajectories remain within the vehicle's physical limits, effectively mitigating cumulative tracking errors typically found in planning-based systems. We evaluate our approach on key CARLA benchmarks, where it significantly outperforms existing state-of-the-art control-based RL experts. On the CARLA Offline Leaderboard v1 and NoCrash benchmarks, our method improves the driving score by 5% and 11%, respectively, and increases the success rate by 8% and 19%.

Problem

• reinforcement learning
• trajectory planning
• autonomous driving
• interpretability
• end-to-end planning

Innovation

• trajectory planning
• reinforcement learning
• Frenet-frame
• kinematic feasibility
• autonomous driving