Learning Safe Autonomous Driving Policies Using Predictive Safety Representations

2025-12-19

AI Summary

Addressing the inherent trade-off between performance optimization and safety constraints in autonomous driving, this paper proposes a reinforcement learning framework grounded in Predictive Safety Representation (PSR), which explicitly models the probability of future constraint violations to guide policy learning. The method integrates safety representation learning, SafeRL optimization, behavior cloning for policy initialization, and multi-source motion prediction modeling. It presents the first systematic evaluation of PSR on real-world driving datasets—WOMD and NuPlan—demonstrating substantial improvements in zero-shot cross-dataset generalization and robustness to observation noise. Experiments show statistically significant gains in task success rate and composite cost metrics (effect size r = 0.65–0.86, p < 0.05). Furthermore, the framework provides an analyzable policy optimizer and characterizes its data distribution adaptation boundary.

Technology Category

• Search and Optimization: Learning to Search
• Planning, Routing, and Scheduling: Learning for Planning and Scheduling
• Natural Language Processing: Safety and Robustness

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Security and Privacy: Large-scale security measurements

Abstract

Safe reinforcement learning (SafeRL) is a prominent paradigm for autonomous driving, where agents are required to optimize performance under strict safety requirements. This dual objective creates a fundamental tension, as overly conservative policies limit driving efficiency while aggressive exploration risks safety violations. The Safety Representations for Safer Policy Learning (SRPL) framework addresses this challenge by equipping agents with a predictive model of future constraint violations and has shown promise in controlled environments. This paper investigates whether SRPL extends to real-world autonomous driving scenarios. Systematic experiments on the Waymo Open Motion Dataset (WOMD) and NuPlan demonstrate that SRPL can improve the reward-safety tradeoff, achieving statistically significant improvements in success rate (effect sizes r = 0.65-0.86) and cost reduction (effect sizes r = 0.70-0.83), with p < 0.05 for observed improvements. However, its effectiveness depends on the underlying policy optimizer and the dataset distribution. The results further show that predictive safety representations play a critical role in improving robustness to observation noise. Additionally, in zero-shot cross-dataset evaluation, SRPL-augmented agents demonstrate improved generalization compared to non-SRPL methods. These findings collectively demonstrate the potential of predictive safety representations to strengthen SafeRL for autonomous driving.

Problem

• Investigates if predictive safety models improve autonomous driving safety-efficiency tradeoffs
• Evaluates SRPL framework on real-world datasets like Waymo and NuPlan
• Assesses robustness and generalization of safety-enhanced reinforcement learning policies

Innovation

• Uses predictive safety representations for safer policy learning
• Improves reward-safety tradeoff in real-world autonomous driving scenarios
• Enhances robustness to observation noise and cross-dataset generalization