AdaWM: Adaptive World Model based Planning for Autonomous Driving

2025-01-22

AI Summary

To address performance degradation in world model–driven reinforcement learning for autonomous driving under the pretraining-finetuning paradigm, this paper proposes an adaptive planning framework. First, a distribution mismatch diagnosis module quantifies inconsistency between the policy and dynamics model in latent space. Second, a mismatch-aware fine-tuning mechanism enables on-demand co-adaptation of policy and model. Third, a LoRA-style low-rank parameter update combined with alignment-oriented lightweight optimization enhances online adaptation efficiency. Evaluated on complex driving tasks in CARLA, our method achieves an average 23.6% improvement in task completion rate and reduces policy transfer failure rate by 57% compared to baseline approaches. These gains demonstrate significantly improved fine-tuning robustness and faster convergence.

Technology Category

• Planning, Routing, and Scheduling: Planning with Language Models
• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

World model based reinforcement learning (RL) has emerged as a promising approach for autonomous driving, which learns a latent dynamics model and uses it to train a planning policy. To speed up the learning process, the pretrain-finetune paradigm is often used, where online RL is initialized by a pretrained model and a policy learned offline. However, naively performing such initialization in RL may result in dramatic performance degradation during the online interactions in the new task. To tackle this challenge, we first analyze the performance degradation and identify two primary root causes therein: the mismatch of the planning policy and the mismatch of the dynamics model, due to distribution shift. We further analyze the effects of these factors on performance degradation during finetuning, and our findings reveal that the choice of finetuning strategies plays a pivotal role in mitigating these effects. We then introduce AdaWM, an Adaptive World Model based planning method, featuring two key steps: (a) mismatch identification, which quantifies the mismatches and informs the finetuning strategy, and (b) alignment-driven finetuning, which selectively updates either the policy or the model as needed using efficient low-rank updates. Extensive experiments on the challenging CARLA driving tasks demonstrate that AdaWM significantly improves the finetuning process, resulting in more robust and efficient performance in autonomous driving systems.

Problem

• Adaptive Learning
• Autonomous Vehicles
• Performance Stability

Innovation

• AdaWM
• Adaptive Environment Modeling
• Autonomous Driving