A Multi-Agent LLM Framework for Design Space Exploration in Autonomous Driving Systems

2025-12-09

AI Summary

Design space exploration (DSE) for autonomous driving systems faces challenges including an exponentially large configuration space, multimodal execution outputs, complex performance trade-offs, and heavy reliance on manual evaluation. Method: This paper proposes a multi-agent large language model (LLM)-based DSE framework that integrates 3D simulation, multimodal perception understanding (vision/text), performance profiling, and automated scheduling to enable end-to-end closed-loop design generation, execution validation, and result analysis. Contributions/Results: (1) A collaborative LLM agent architecture is introduced, supporting semantic configuration modeling and automatic interpretation of multimodal execution feedback; (2) Correctness assessment and Pareto-front optimization are achieved without human annotation. Evaluated on a Robotaxi case study, the framework discovers more Pareto-optimal configurations than genetic algorithms under identical exploration budgets, reducing average navigation time by 23.6% and significantly improving both DSE efficiency and solution quality.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Planning, Routing, and Scheduling: Planning with Language Models
• Multiagent Systems: Multiagent Planning

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Agentic search

Abstract

Designing autonomous driving systems requires efficient exploration of large hardware/software configuration spaces under diverse environmental conditions, e.g., with varying traffic, weather, and road layouts. Traditional design space exploration (DSE) approaches struggle with multi-modal execution outputs and complex performance trade-offs, and often require human involvement to assess correctness based on execution outputs. This paper presents a multi-agent, large language model (LLM)-based DSE framework, which integrates multi-modal reasoning with 3D simulation and profiling tools to automate the interpretation of execution outputs and guide the exploration of system designs. Specialized LLM agents are leveraged to handle user input interpretation, design point generation, execution orchestration, and analysis of both visual and textual execution outputs, which enables identification of potential bottlenecks without human intervention. A prototype implementation is developed and evaluated on a robotaxi case study (an SAE Level 4 autonomous driving application). Compared with a genetic algorithm baseline, the proposed framework identifies more Pareto-optimal, cost-efficient solutions with reduced navigation time under the same exploration budget. Experimental results also demonstrate the efficiency of the adoption of the LLM-based approach for DSE. We believe that this framework paves the way to the design automation of autonomous driving systems.

Problem

• Automates design space exploration for autonomous driving systems
• Handles multi-modal outputs and complex trade-offs without human intervention
• Identifies Pareto-optimal solutions efficiently using LLM agents

Innovation

• Multi-agent LLM framework automates design space exploration
• Integrates multimodal reasoning with simulation and profiling tools
• Specialized agents interpret outputs and identify bottlenecks autonomously