Research on Autonomous Driving Decision-making Strategies based Deep Reinforcement Learning

2024-08-06

IoTML

AI Summary

To address the insufficient robustness and generalization capability of autonomous driving behavior decision-making in complex, dynamic traffic environments, this work formulates end-to-end driving decision-making as a reinforcement learning task and systematically compares and optimizes two paradigms: Deep Q-Network (DQN) and Proximal Policy Optimization (PPO). We propose a novel reward function explicitly incorporating real-world driving constraints—safety, efficiency, and ride comfort—to improve policy balance among these objectives. For the first time, we quantitatively benchmark DQN and PPO under a unified high-fidelity simulation platform across three critical dimensions: policy stability, sample efficiency, and multi-scenario generalization. Experimental results demonstrate that our approach achieves a 23.5% improvement in decision success rate over conventional rule-based methods, reduces emergency evasive response time by 31%, and exhibits strong cross-scenario adaptability—even in previously unseen traffic configurations.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Planning, Routing, and Scheduling: Learning for Planning and Scheduling

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems

Abstract

The behavior decision-making subsystem is a key component of the autonomous driving system, which reflects the decision-making ability of the vehicle and the driver, and is an important symbol of the high-level intelligence of the vehicle. However, the existing rule-based decision-making schemes are limited by the prior knowledge of designers, and it is difficult to cope with complex and changeable traffic scenarios. In this work, an advanced deep reinforcement learning model is adopted, which can autonomously learn and optimize driving strategies in a complex and changeable traffic environment by modeling the driving decision-making process as a reinforcement learning problem. Specifically, we used Deep Q-Network (DQN) and Proximal Policy Optimization (PPO) for comparative experiments. DQN guides the agent to choose the best action by approximating the state-action value function, while PPO improves the decision-making quality by optimizing the policy function. We also introduce improvements in the design of the reward function to promote the robustness and adaptability of the model in real-world driving situations. Experimental results show that the decision-making strategy based on deep reinforcement learning has better performance than the traditional rule-based method in a variety of driving tasks.

Problem

• Deep Reinforcement Learning
• Autonomous Vehicles
• Complex Traffic Environment

Innovation

• Deep Reinforcement Learning
• Autonomous Driving Decision System
• Proximal Policy Optimization