Knowledge-Data-Dual-Driven Reinforcement Learning for Autonomous Vehicle Control in Mixed Traffic

2026-08-13

AI Summary

This study addresses the challenges of intent recognition, insufficient long-tail safety exploration, and training instability in mixed traffic environments by proposing a knowledge-data dual-driven reinforcement learning framework. The method integrates generative intent prediction with physical constraints to construct intent-aware state representations, while employing shared embedding coupling modules and asynchronous multi-timescale optimization algorithms to enable efficient and safe exploration. Experimental results demonstrate that this framework effectively mitigates intent uncertainty and accelerates convergence, significantly outperforming traditional baselines in safety, traffic efficiency, and ride comfort. Consequently, this work establishes a novel paradigm for autonomous driving in complex scenarios.

Abstract

In mixed traffic, decision-making for autonomous vehicles (AVs) confronts three interrelated challenges. First, physics-based priors incorporated into reinforcement learning (RL) models fail to capture latent interactive vehicle intentions and diverse driver behaviors, limiting the proactive reasoning capabilities. Second, abrupt maneuvers by surrounding vehicles cause non-stationarity, leaving long-tail safety events under-explored. Third, hybrid action spaces destabilize unified RL training due to the different temporal scales of continuous car-following and discrete lane-changing maneuvers. To address these issues, we propose Knowledge-Data Dual-driven Reinforcement Learning (KDDRL). First, a conditional deep generative model synthesizes intention-aware future trajectories, converting passive perception into proactive predictive states. Second, a knowledge-data dual-driven paradigm operates on these predictive states, fusing probabilistic data-driven insights with physical constraints to guide safe exploration through safety-critical scenarios. Third, a coupling module compresses both intention-aware trajectories and physical constraints into compact shared embeddings. This unified representation enables asynchronous multi-timescale optimization of continuous car-following and discrete lane-changing while preserving mutual information. Evaluations on dataset-calibrated simulations demonstrate that KDDRL effectively handles intention uncertainty, accelerates training convergence, and outperforms conventional baseline methods in terms of safety, efficiency, and comfort.

Problem

• Mixed Traffic
• Autonomous Vehicle Decision-Making
• Intention Uncertainty
• Non-stationarity
• Hybrid Action Space

Innovation

• Knowledge-Data Dual-driven RL
• Intention-aware Trajectory Generation
• Asynchronous Multi-timescale Optimization
• Mixed Traffic Decision-making
• Shared Embeddings