Highly Accurate and Diverse Traffic Data: The DeepScenario Open 3D Dataset

2025-04-24

AI Summary

Conventional onboard 3D trajectory datasets suffer from occlusions and limited field-of-view, hindering accurate modeling of distant traffic participants. To address this, we introduce DSC3D—the first open, occlusion-free, high-precision 6DoF monocular UAV-based 3D trajectory dataset—featuring 175K+ multi-class trajectories across five representative traffic scenarios. We propose the first end-to-end monocular UAV 3D tracking pipeline, uniquely covering long-tail scenarios such as complex human-vehicle interactions and full-cycle parking. Accuracy is ensured via monocular visual SLAM, multi-object trajectory optimization, UAV collaborative calibration, and multi-sensor spatiotemporal alignment. We release the dataset alongside an interactive visualization platform. Extensive evaluation demonstrates DSC3D’s effectiveness on motion prediction, behavioral modeling, and safety verification tasks, enabling research on generative, reactive traffic agents.

Technology Category

• Computer Vision: 3D Computer Vision
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Security and Privacy: Data transparency and provenance

Abstract

Accurate 3D trajectory data is crucial for advancing autonomous driving. Yet, traditional datasets are usually captured by fixed sensors mounted on a car and are susceptible to occlusion. Additionally, such an approach can precisely reconstruct the dynamic environment in the close vicinity of the measurement vehicle only, while neglecting objects that are further away. In this paper, we introduce the DeepScenario Open 3D Dataset (DSC3D), a high-quality, occlusion-free dataset of 6 degrees of freedom bounding box trajectories acquired through a novel monocular camera drone tracking pipeline. Our dataset includes more than 175,000 trajectories of 14 types of traffic participants and significantly exceeds existing datasets in terms of diversity and scale, containing many unprecedented scenarios such as complex vehicle-pedestrian interaction on highly populated urban streets and comprehensive parking maneuvers from entry to exit. DSC3D dataset was captured in five various locations in Europe and the United States and include: a parking lot, a crowded inner-city, a steep urban intersection, a federal highway, and a suburban intersection. Our 3D trajectory dataset aims to enhance autonomous driving systems by providing detailed environmental 3D representations, which could lead to improved obstacle interactions and safety. We demonstrate its utility across multiple applications including motion prediction, motion planning, scenario mining, and generative reactive traffic agents. Our interactive online visualization platform and the complete dataset are publicly available at app.deepscenario.com, facilitating research in motion prediction, behavior modeling, and safety validation.

Problem

• Addresses occlusion and limited coverage in traditional 3D traffic datasets
• Provides diverse, high-quality 6DoF trajectories for autonomous driving
• Enables complex scenario modeling like urban interactions and parking

Innovation

• Monocular camera drone tracking pipeline
• Occlusion-free 6DOF trajectory dataset
• Diverse urban and highway scenarios