How Far Can 5,500 Hours of Driving Take You? A Scaling Law Analysis of Video Diffusion Models

2026-08-28

AI Summary

研究通过分析不同规模的视频扩散模型在5,500小时驾驶数据上的表现，发现延长训练比扩大模型规模更有效提高性能，但大模型仍有优势。

Abstract

Video generation for autonomous driving cannot follow the web-scale route: driving data is expensive to collect, bound by privacy requirements, and cannot be scraped at will, so models must make the most of a fixed corpus. We present a systematic scaling-law study of video diffusion models trained from scratch on driving data: a family of models from 1M to 9B parameters, trained at different exposures on up to 5,500 hours of driving. Validation loss follows consistent power laws in both model size and training exposure, answering the questions that shape a training budget: whether compute is better spent on longer training or on a larger model, and whether more data is needed. Loss improves much faster with training exposure than with model size, making longer training the most effective way to improve a fixed model under limited compute. However, larger models continue to achieve lower asymptotic loss, so compute-optimal scaling still favors increasing model size when sufficient compute and data are available. Guided by these laws, we train a 9B-parameter model, to our knowledge the largest video diffusion model trained from scratch on driving data: it sets a new open-source state of the art for driving video generation, as measured on nuScenes. Our code and pretrained models are available at https://github.com/valeoai/VATIX. NATIX is separately releasing the underlying driving data in stages.

Problem

• video generation
• autonomous driving
• scaling laws
• training exposure
• model size

Innovation

• scaling law
• video diffusion models
• driving data
• training exposure
• model size