MV2: Multi-View Multi-Vehicle Driving Dataset for Novel View Synthesis

2026-08-12

AI Summary

Existing novel view synthesis methods are limited in real-world driving scenarios by sparse viewpoints, dynamic objects, and single-trajectory data. To address these challenges, this work introduces a multi-view, multi-vehicle urban driving dataset—captured synchronously from cars, scooters, and drones—that enables, for the first time, large-baseline image acquisition across vehicles and trajectories, accompanied by high-precision poses and pixel-level annotations. Sequences are registered via Structure-from-Motion (SfM) and refined with manually verified correspondences to support evaluation of differentiable rendering and novel view synthesis algorithms. Comprising 12,000 images across 50 scenes, the dataset’s benchmark experiments reveal the critical impact of viewpoint span on synthesis quality and highlight performance gaps in current pose estimation methods.

Abstract

Differentiable rendering has advanced novel view synthesis (NVS), yet applying it to real-world driving remains difficult due to sparse capture viewpoints, dynamic objects, and limited multi-trajectory data. We introduce the Multi-View Multi-Vehicle (MV2) dataset and benchmark for evaluating NVS models under large viewpoint changes in dynamic urban scenes. MV2 features synchronized captures from a car, scooter, and drone, each following distinct yet synchronized trajectories. Training NVS methods on one vehicle's camera stream and testing on another enables evaluation under substantially larger viewpoint variations than existing single-trajectory datasets. All sequences are registered via Structure-from-Motion and camera poses verified using manual pixel-level correspondence annotations, yielding 50 high-quality scenes with 12000 images. Benchmarking recent NVS and camera pose estimation methods shows that NVS performance degrades with increasing viewpoint disparity, and that feed-forward pose estimators notably lag behind optimization-based approaches, highlighting MV2 as a rigorous testbed for NVS in driving. The dataset, benchmark protocol, and project resources are available at https://mv2-dataset.github.io/.

Problem

• novel view synthesis
• multi-view
• dynamic scenes
• viewpoint disparity
• driving dataset

Innovation

• novel view synthesis
• multi-view dataset
• multi-vehicle trajectories
• dynamic urban scenes
• camera pose estimation