SimpleVSF: VLM-Scoring Fusion for Trajectory Prediction of End-to-End Autonomous Driving

2025-10-20

AI Summary

To address the insufficient robustness of end-to-end autonomous driving in complex scenarios—particularly in trajectory decision-making—this paper proposes SimpleVSF, a novel framework that introduces visual-language models (VLMs) into the trajectory scoring stage for the first time, enabling cognitive-level scene understanding. We design a two-tier fusion mechanism that jointly integrates quantitative weighted fusion with qualitative, context-aware reasoning to holistically optimize safety, comfort, and efficiency. By eliminating redundant components, SimpleVSF achieves high performance with a lightweight architecture, significantly improving planning reliability under long-horizon and highly dynamic conditions. Evaluated on the ICCV 2025 NAVSIM v2 Challenge, SimpleVSF attains state-of-the-art (SOTA) overall performance, demonstrating superior balance across multiple metrics compared to existing approaches.

Technology Category

• Planning, Routing, and Scheduling: Planning with Language Models
• Computer Vision: Language and Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

End-to-end autonomous driving has emerged as a promising paradigm for achieving robust and intelligent driving policies. However, existing end-to-end methods still face significant challenges, such as suboptimal decision-making in complex scenarios. In this paper,we propose SimpleVSF (Simple VLM-Scoring Fusion), a novel framework that enhances end-to-end planning by leveraging the cognitive capabilities of Vision-Language Models (VLMs) and advanced trajectory fusion techniques. We utilize the conventional scorers and the novel VLM-enhanced scorers. And we leverage a robust weight fusioner for quantitative aggregation and a powerful VLM-based fusioner for qualitative, context-aware decision-making. As the leading approach in the ICCV 2025 NAVSIM v2 End-to-End Driving Challenge, our SimpleVSF framework demonstrates state-of-the-art performance, achieving a superior balance between safety, comfort, and efficiency.

Problem

• Enhancing decision-making in complex autonomous driving scenarios
• Integrating Vision-Language Models for improved trajectory prediction
• Balancing safety, comfort, and efficiency in end-to-end planning

Innovation

• Fuses vision-language models with trajectory scoring
• Combines quantitative and qualitative fusion techniques
• Enhances planning using VLM-based context-aware decision-making