Reward-Driven Automated Curriculum Learning for Interaction-Aware Self-Driving at Unsignalized Intersections

2024-03-20

IEEE/RJS International Conference on Intelligent RObots and Systems

AI Summary

To address the interactive decision-making challenge under dual uncertainty—regarding both the number and intentions of surrounding vehicles—at unsignalized intersections, this paper proposes a reward-driven automated curriculum learning framework. Methodologically, it introduces a progressive curriculum (incrementally increasing the number of interacting vehicles), an intention-aware fine-grained reward function, and a novel dynamic-weighted automatic curriculum selection mechanism. The approach integrates deep reinforcement learning with explicit intention modeling and is rigorously evaluated on two benchmark platforms: Highway-Env and CARLA. In Highway-Env, it achieves state-of-the-art task success rates, demonstrates strong robustness to curriculum initialization, and exhibits excellent cross-scenario generalization. In CARLA, it validates high driving safety and effective interaction in photorealistic, high-fidelity traffic environments. The core contribution is the first automated curriculum learning paradigm specifically designed for interaction under intention and cardinality uncertainty, coupled with a transferable decision-making architecture.

Technology Category

• Reasoning under Uncertainty: Sequential Decision Making
• Planning, Routing, and Scheduling: Learning for Planning and Scheduling
• Multiagent Systems: Adversarial Agents

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

In this work, we present a reward-driven automated curriculum reinforcement learning approach for interaction-aware self-driving at unsignalized intersections, taking into account the uncertainties associated with surrounding vehicles (SVs). These uncertainties encompass the uncertainty of SVs’ driving intention and also the quantity of SVs. To deal with this problem, the curriculum set is specifically designed to accommodate a progressively increasing number of SVs. By implementing an automated curriculum selection mechanism, the importance weights are rationally allocated across various curricula, thereby facilitating improved sample efficiency and training outcomes. Furthermore, the reward function is meticulously designed to guide the agent towards effective policy exploration. Thus the proposed framework could proactively address the above uncertainties at unsignalized intersections by employing the automated curriculum learning technique that progressively increases task difficulty, and this ensures safe self-driving through effective interaction with SVs. Comparative experiments are conducted in Highway_Env, and the results indicate that our approach achieves the highest task success rate, attains strong robustness to initialization parameters of the curriculum selection module, and exhibits superior adaptability to diverse situational configurations at unsignalized intersections. Furthermore, the effectiveness of the proposed method is validated using the high-fidelity CARLA simulator.

Problem

• Autonomous Vehicles
• Intersection Management
• Vehicular Interaction

Innovation

• Adaptive Learning Framework
• Autonomous Vehicle Interaction
• Progressive Curriculum Design