SACA: A Scenario-Aware Collision Avoidance Framework for Autonomous Vehicles Integrating LLMs-Driven Reasoning

2025-03-31

AI Summary

To address the challenges of high inference latency, poor robustness, and difficulty in simultaneously satisfying legal, ethical, and affective constraints in real-time autonomous collision avoidance under extreme scenarios, this paper proposes the first LLM-driven “scene-preview” decision-making framework. The method integrates obstacle reachability analysis, motion-intent prediction, memory-augmented fine-tuned LLM online inference, scene similarity retrieval, and a precomputed policy library to achieve low-latency, socially acceptable collision avoidance decisions. Its key innovations lie in decoupling offline ethical/legal evaluation from online lightweight inference and introducing a memory bank to enable dynamic contextual adaptation. Real-vehicle experiments demonstrate significant reductions in collision-related losses under extreme high-risk conditions and a 37.2% decrease in false triggering rates in complex environments.

Technology Category

• Search and Optimization: Learning to Search
• Planning, Routing, and Scheduling: Planning with Language Models
• Natural Language Processing: Safety and Robustness

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Reliable collision avoidance under extreme situations remains a critical challenge for autonomous vehicles. While large language models (LLMs) offer promising reasoning capabilities, their application in safety-critical evasive maneuvers is limited by latency and robustness issues. Even so, LLMs stand out for their ability to weigh emotional, legal, and ethical factors, enabling socially responsible and context-aware collision avoidance. This paper proposes a scenario-aware collision avoidance (SACA) framework for extreme situations by integrating predictive scenario evaluation, data-driven reasoning, and scenario-preview-based deployment to improve collision avoidance decision-making. SACA consists of three key components. First, a predictive scenario analysis module utilizes obstacle reachability analysis and motion intention prediction to construct a comprehensive situational prompt. Second, an online reasoning module refines decision-making by leveraging prior collision avoidance knowledge and fine-tuning with scenario data. Third, an offline evaluation module assesses performance and stores scenarios in a memory bank. Additionally, A precomputed policy method improves deployability by previewing scenarios and retrieving or reasoning policies based on similarity and confidence levels. Real-vehicle tests show that, compared with baseline methods, SACA effectively reduces collision losses in extreme high-risk scenarios and lowers false triggering under complex conditions. Project page: https://sean-shiyuez.github.io/SACA/.

Problem

• Enhancing collision avoidance in extreme scenarios for autonomous vehicles
• Integrating LLMs for ethical and context-aware decision-making in collisions
• Improving deployability and robustness of safety-critical evasive maneuvers

Innovation

• Predictive scenario analysis for situational awareness
• Online reasoning with prior knowledge and fine-tuning
• Precomputed policy method for scenario preview