Efficient and Generalized end-to-end Autonomous Driving System with Latent Deep Reinforcement Learning and Demonstrations

2024-01-22

AI Summary

To address the high sample complexity, poor generalization, and insufficient safety guarantees of existing end-to-end autonomous driving approaches, this paper proposes EGADS—a latent-space deep reinforcement learning framework tailored for complex urban driving scenarios. Methodologically, EGADS integrates variational inference with normalizing flows to establish a prior-free RL paradigm; introduces a robust safety-constrained decision-making mechanism to ensure reliability; and constructs a professional driving demonstration dataset collected via a Logitech G29 steering wheel. Leveraging latent-space modeling, VAE-based representation learning, and joint imitation-reinforcement optimization, EGADS substantially reduces training sample requirements. Empirical evaluation demonstrates significant improvements in safety performance within urban environments and superior cross-scenario generalization compared to state-of-the-art end-to-end methods.

Technology Category

• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

An intelligent driving system should dynamically formulate appropriate driving strategies based on the current environment and vehicle status while ensuring system security and reliability. However, methods based on reinforcement learning and imitation learning often suffer from high sample complexity, poor generalization, and low safety. To address these challenges, this paper introduces an Efficient and Generalized end-to-end Autonomous Driving System (EGADS) for complex and varied scenarios. The RL agent in our EGADS combines variational inference with normalizing flows, which are independent of distribution assumptions. This combination allows the agent to capture historical information relevant to driving in latent space effectively, thereby significantly reducing sample complexity. Additionally, we enhance safety by formulating robust safety constraints and improve generalization and performance by integrating RL with expert demonstrations. Experimental results demonstrate that, compared to existing methods, EGADS significantly reduces sample complexity, greatly improves safety performance, and exhibits strong generalization capabilities in complex urban scenarios. Particularly, we contributed an expert dataset collected through human expert steering wheel control, specifically using the G29 steering wheel.

Problem

• High sample complexity in reinforcement learning for autonomous driving
• Poor generalization of driving systems in varied scenarios
• Low safety performance in end-to-end autonomous driving

Innovation

• Combines variational inference with normalizing flows
• Integrates RL with expert demonstrations
• Formulates robust safety constraints