Risk-Aware Reinforcement Learning for Autonomous Driving: Improving Safety When Driving through Intersection

2025-03-25

AI Summary

To address insufficient safety modeling in classical reinforcement learning for autonomous driving at unsignalized intersections, this paper proposes a risk-aware Actor-Critic framework. Methodologically: (1) a dual-critic architecture jointly leverages safety constraints and reward signals to guide policy optimization; (2) safety-aware action projection is achieved via Lagrangian relaxation and cyclic gradient iteration over the action space; (3) explicit driving risk modeling, safety-constrained action projection, and a novel multi-hop, multi-layer perception hybrid attention mechanism (MMAM) are co-embedded into the policy network—constituting the first such integration. Experiments in high-fidelity simulation demonstrate substantial reductions in collision rates and notable improvements in traffic throughput. Ablation studies confirm that both explicit risk modeling and MMAM critically enhance safety assurance and decision quality, while preserving robustness and dynamic adaptability under varying traffic conditions.

Technology Category

• Natural Language Processing: Safety and Robustness
• Search and Optimization: Learning to Search
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Applying reinforcement learning to autonomous driving has garnered widespread attention. However, classical reinforcement learning methods optimize policies by maximizing expected rewards but lack sufficient safety considerations, often putting agents in hazardous situations. This paper proposes a risk-aware reinforcement learning approach for autonomous driving to improve the safety performance when crossing the intersection. Safe critics are constructed to evaluate driving risk and work in conjunction with the reward critic to update the actor. Based on this, a Lagrangian relaxation method and cyclic gradient iteration are combined to project actions into a feasible safe region. Furthermore, a Multi-hop and Multi-layer perception (MLP) mixed Attention Mechanism (MMAM) is incorporated into the actor-critic network, enabling the policy to adapt to dynamic traffic and overcome permutation sensitivity challenges. This allows the policy to focus more effectively on surrounding potential risks while enhancing the identification of passing opportunities. Simulation tests are conducted on different tasks at unsignalized intersections. The results show that the proposed approach effectively reduces collision rates and improves crossing efficiency in comparison to baseline algorithms. Additionally, our ablation experiments demonstrate the benefits of incorporating risk-awareness and MMAM into RL.

Problem

• Improving intersection safety in autonomous driving using risk-aware RL
• Addressing safety gaps in classical RL for hazardous driving scenarios
• Enhancing collision avoidance and efficiency at unsignalized intersections

Innovation

• Risk-aware RL with safe critics for intersection safety
• Lagrangian relaxation and cyclic gradient for safe actions
• MMAM in actor-critic for dynamic traffic adaptation