SARAD: LLM-Based Safety-Aware Hybrid Reinforcement Learning with Collision Prediction for Autonomous Driving

2026-05-27

AI Summary

This work proposes SARAD, a novel framework that integrates large language models (LLMs) with deep reinforcement learning (DRL) to address the challenge of balancing safety and efficiency in autonomous driving decision-making. SARAD leverages retrieval-augmented generation (RAG) to produce guided actions from a dynamic expert knowledge base, replacing unsafe random exploration during training. An attention-based discriminator incorporates LLM-derived prior knowledge into policy optimization, while a dedicated collision prediction module—fine-tuned on historical collision data—further enhances safety. Evaluated in the Highway-Env simulation environment, SARAD demonstrates significantly improved driving performance by simultaneously ensuring safety and increasing decision-making efficiency, thereby validating its effectiveness.

Abstract

Ensuring both safety and efficiency in decision-making for autonomous driving systems remains a fundamental challenge. Traditional Deep Reinforcement Learning (DRL) suffers from unsafe random exploration and slow convergence, while Large Language Models (LLMs) demonstrate inherent latency in real-time inference operations. To address these limitations, this paper proposes SARAD, a novel safety-aware hybrid framework that synergizes LLMs and DRL for autonomous driving. SARAD substitutes the random exploration of DRL with Retrieval-Augmented Generation (RAG)-enhanced, LLM-guided decisions sourced from a dynamic expert knowledge repository. An attention discriminator is proposed to integrate the prior knowledge of LLMs into DRL policy optimization. A collision predictor module, fine-tuned with historical collision data, is further designed to improve vehicle safety. Extensive experiments show that SARAD achieves significant performance improvements in the Highway-Env simulator, validating the effectiveness of the proposed model in autonomous driving.

Problem

• autonomous driving
• safety
• reinforcement learning
• Large Language Models
• collision prediction

Innovation

• Hybrid Reinforcement Learning
• Large Language Models
• Retrieval-Augmented Generation
• Collision Prediction
• Safety-Aware Decision Making