Metis: A Generalizable and Efficient World-Action Model for Autonomous Driving and Urban Navigation

2026-06-14

AI Summary

Existing world action models suffer from high inference latency, representation mismatch, and limited generalization due to the tight coupling of video generation and action prediction. This work proposes Metis, an end-to-end decoupled framework that separates these two tasks through a hybrid Transformer architecture augmented with an asymmetric attention mask. This design enables action prediction during inference without explicitly generating future video frames, thereby preserving training consistency while significantly improving inference efficiency and generalization. Experimental results demonstrate that Metis achieves state-of-the-art performance on the NAVSIM (navhard/navtest) and CityWalker benchmarks. Furthermore, real-world deployment validates its practicality and computational efficiency in actual robotic systems.

Abstract

World action models~(WAMs) have shown great promise for autonomous driving and urban navigation. Built upon Vision-Language-Action models or video generation models, existing approaches suffer key limitations: (1) High inference latency due to future observation prediction at test time, and (2) tightly coupled video and action modeling leading to representational mismatch and degraded generalization. To address both issues, we propose Metis, an end-to-end WAM framework that decouples video generation and action prediction. Specifically, Metis employs a Mixture-of-Transformers architecture with dedicated experts for video generation and action prediction, preserving the intrinsic distributional properties of each task. To enhance efficiency, we introduce an asymmetric attention mask that enables joint training of both experts while allowing the action model to bypass explicit video generation during inference. This design ensures training-inference consistency and significantly reduces computational costs without compromising planning performance. Extensive experiments demonstrate state-of-the-art performance on the NAVSIM navhard and navtest benchmarks and the CityWalker navigation benchmark, validating both the generalizability and efficiency across diverse tasks. Real-robot deployments further confirm the practical feasibility of our approach.

Problem

• world action models
• autonomous driving
• urban navigation
• inference latency
• representational mismatch

Innovation

• World Action Model
• Decoupled Architecture
• Mixture-of-Transformers
• Asymmetric Attention Mask
• Efficient Inference