Drive My Way: Preference Alignment of Vision-Language-Action Model for Personalized Driving

2026-03-26

AI Summary

This work addresses the limitations of existing end-to-end autonomous driving systems, which struggle to model individual driving preferences and lack joint understanding of long-term behavioral habits and natural language instructions. To overcome these challenges, we propose Drive My Way (DMW), a novel framework that integrates personalized driving styles—encoded via user-specific embeddings—with real-time natural language commands to construct a vision-language-action conditional policy. By unifying visual perception, linguistic intent, and personalized behavior, DMW enables adaptive driving aligned with individual user characteristics. Evaluated on the Bench2Drive benchmark, our approach demonstrates superior style adaptability, and user studies confirm that its generated driving behaviors are consistently recognized as reflecting specific individuals’ driving styles, thereby advancing human-centered intelligent driving systems.

Technology Category

• Humans and AI: Human-Aware Planning and Behavior Prediction
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Cognitive Modeling & Cognitive Systems: Adaptive Behavior

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Human driving behavior is inherently personal, which is shaped by long-term habits and influenced by short-term intentions. Individuals differ in how they accelerate, brake, merge, yield, and overtake across diverse situations. However, existing end-to-end autonomous driving systems either optimize for generic objectives or rely on fixed driving modes, lacking the ability to adapt to individual preferences or interpret natural language intent. To address this gap, we propose Drive My Way (DMW), a personalized Vision-Language-Action (VLA) driving framework that aligns with users' long-term driving habits and adapts to real-time user instructions. DMW learns a user embedding from our personalized driving dataset collected across multiple real drivers and conditions the policy on this embedding during planning, while natural language instructions provide additional short-term guidance. Closed-loop evaluation on the Bench2Drive benchmark demonstrates that DMW improves style instruction adaptation, and user studies show that its generated behaviors are recognizable as each driver's own style, highlighting personalization as a key capability for human-centered autonomous driving. Our data and code are available at https://dmw-cvpr.github.io/.

Problem

• personalized driving
• preference alignment
• vision-language-action model
• autonomous driving
• user-specific behavior

Innovation

• personalized autonomous driving
• vision-language-action model
• user embedding
• natural language instruction
• driving style adaptation