DiffVLA++: Bridging Cognitive Reasoning and End-to-End Driving through Metric-Guided Alignment

2025-10-20

AI Summary

Traditional end-to-end (E2E) driving models yield physically feasible trajectories but suffer from poor generalization, whereas vision-language-action (VLA) models excel in semantic understanding yet often produce non-physical trajectories. To bridge this gap, we propose Metric-guided Trajectory Alignment (MTA), the first framework to achieve dynamic alignment between VLA and E2E modules directly in trajectory space. Specifically, the VLA module generates semantically rich candidate trajectories, while the E2E module provides densely sampled, physically feasible ones; a metric-guided scoring mechanism jointly optimizes semantic coherence and physical plausibility. MTA unifies world knowledge and motion constraints within a single differentiable architecture. Evaluated on the ICCV 2025 Autonomous Driving Challenge, our method achieves an EPDMS score of 49.12—marking a significant improvement in safety and robustness, particularly for long-tail and complex urban driving scenarios.

Technology Category

• Computer Vision: Large Vision Models
• Planning, Routing, and Scheduling: Planning with Language Models
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Conventional end-to-end (E2E) driving models are effective at generating physically plausible trajectories, but often fail to generalize to long-tail scenarios due to the lack of essential world knowledge to understand and reason about surrounding environments. In contrast, Vision-Language-Action (VLA) models leverage world knowledge to handle challenging cases, but their limited 3D reasoning capability can lead to physically infeasible actions. In this work we introduce DiffVLA++, an enhanced autonomous driving framework that explicitly bridges cognitive reasoning and E2E planning through metric-guided alignment. First, we build a VLA module directly generating semantically grounded driving trajectories. Second, we design an E2E module with a dense trajectory vocabulary that ensures physical feasibility. Third, and most critically, we introduce a metric-guided trajectory scorer that guides and aligns the outputs of the VLA and E2E modules, thereby integrating their complementary strengths. The experiment on the ICCV 2025 Autonomous Grand Challenge leaderboard shows that DiffVLA++ achieves EPDMS of 49.12.

Problem

• Bridging cognitive reasoning and end-to-end driving
• Ensuring physical feasibility of autonomous trajectories
• Aligning semantic understanding with metric-guided planning

Innovation

• VLA module generates semantic driving trajectories
• E2E module ensures physical feasibility with dense vocabulary
• Metric-guided scorer aligns cognitive and physical outputs