OVPD: A Virtual-Physical Fusion Testing Dataset of OnSite Auton-omous Driving Challenge

2026-04-22

AI Summary

Existing autonomous driving test datasets often lack realistic vehicle dynamics and closed-loop traffic interactions, limiting their ability to assess deployment readiness of algorithms. This work proposes a closed-loop, hybrid simulation-testing paradigm centered on real-vehicle-in-the-loop (RVIL) testing, integrating digital twin rendering, multimodal sensor fusion, and vehicle-infrastructure cooperative perception within a professional proving ground. The resulting framework establishes a high-fidelity, reproducible, and diagnosable evaluation environment. As part of this effort, the authors release the OVPD dataset, which includes data from 20 competing teams, 15 atomic scenario types, and nearly three hours of multimodal recordings. This resource enables multidimensional closed-loop evaluation across safety, efficiency, and comfort metrics, while also supporting validation on long-tail scenarios, thereby addressing a critical gap in high-fidelity autonomous driving test data.

Technology Category

• Application Domains: Mobility, Driving & Flight
• Search and Optimization: Sampling/Simulation-based Search
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization
• Security and Privacy: Data transparency and provenance
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

The rapid iteration of autonomous driving algorithms has created a growing demand for high-fidelity, replayable, and diagnosable testing data. However, many public datasets lack real vehicle dynamics feedback and closed-loop interaction with surrounding traffic and road infrastructure, limiting their ability to reflect deployment readiness. To address this gap, we present OVPD (OnSite Virtual-Physical Dataset), a virtual-physical fusion testing dataset released from the 2025 OnSite Autonomous Driving Challenge. Centered on real-vehicle-in-the-loop testing, OVPD integrates virtual background traffic with vehicle-infrastructure perception to build controllable and interactive closed-loop test environments on a proving ground. The dataset contains 20 testing clips from 20 teams over a scenario chain of 15 atomic scenarios, totaling nearly 3 hours of multi-modal data, including vehicle trajectories and states, control commands, and digital-twin-rendered surround-view observations. OVPD supports long-tail planning and decision-making validation, open-loop or platform-enabled closed-loop evaluation, and comprehensive assessment across safety, efficiency, comfort, rule compliance, and traffic impact, providing actionable evidence for failure diagnosis and iterative improvement. The dataset is available via: https://huggingface.co/datasets/Yuhang253820/Onsite_OPVD

Problem

• autonomous driving
• testing dataset
• vehicle dynamics
• closed-loop interaction
• deployment readiness

Innovation

• virtual-physical fusion
• real-vehicle-in-the-loop
• closed-loop testing
• digital twin
• autonomous driving dataset