LLM-Driven Scenario-Aware Planning for Autonomous Driving

2026-01-29

AI Summary

This work addresses the challenge of achieving both high-speed efficiency and safe maneuvering in dense traffic, where existing hybrid planners often fall short due to their reliance on heuristic scene recognition and low-frequency control updates, leading to unreliable mode transitions and suboptimal throughput. To overcome these limitations, the authors propose LAP, a novel framework that leverages a large language model (LLM) for semantic scene understanding in autonomous driving and integrates its reasoning outputs into a joint optimization scheme for mode selection and trajectory planning. By dynamically switching between high-speed and precision driving modes, LAP enables more adaptive and robust navigation. The method combines LLM-based reasoning with tree-search model predictive control (MPC) and an alternating minimization algorithm, implemented in ROS using Python. High-fidelity simulations demonstrate that LAP significantly outperforms current baselines in both travel time and task success rate.

Technology Category

• Planning, Routing, and Scheduling: Planning with Language Models
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Hybrid planner switching framework (HPSF) for autonomous driving needs to reconcile high-speed driving efficiency with safe maneuvering in dense traffic. Existing HPSF methods often fail to make reliable mode transitions or sustain efficient driving in congested environments, owing to heuristic scene recognition and low-frequency control updates. To address the limitation, this paper proposes LAP, a large language model (LLM) driven, adaptive planning method, which switches between high-speed driving in low-complexity scenes and precise driving in high-complexity scenes, enabling high qualities of trajectory generation through confined gaps. This is achieved by leveraging LLM for scene understanding and integrating its inference into the joint optimization of mode configuration and motion planning. The joint optimization is solved using tree-search model predictive control and alternating minimization. We implement LAP by Python in Robot Operating System (ROS). High-fidelity simulation results show that the proposed LAP outperforms other benchmarks in terms of both driving time and success rate.

Problem

• autonomous driving
• hybrid planner switching
• scene recognition
• trajectory generation
• congested environments

Innovation

• Large Language Model (LLM)
• Scenario-Aware Planning
• Hybrid Planner Switching
• Joint Optimization
• Model Predictive Control