Correcting and Quantifying Systematic Errors in 3D Box Annotations for Autonomous Driving

2026-01-20

AI Summary

This work addresses a systematic error in 3D bounding box annotations within dynamic scenes, caused by temporal misalignment among sensor scans, which displaces annotations from the true physical trajectories of objects. We are the first to identify and quantify this error in major autonomous driving datasets and propose an offline optimization method grounded in physical motion models to enforce spatiotemporal consistency across LiDAR time-series data. Evaluated on Argoverse 2, MAN TruckScenes, and a custom dataset, our approach corrects annotation offsets as large as 2.5 meters, improving annotation quality by over 17%. Notably, the impact of this error on benchmark evaluations exceeds the performance gains of typical state-of-the-art methods, underscoring its significant confounding effect on algorithm assessment.

Technology Category

• Computer Vision: Motion & Tracking
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Accurate ground truth annotations are critical to supervised learning and evaluating the performance of autonomous vehicle systems. These vehicles are typically equipped with active sensors, such as LiDAR, which scan the environment in predefined patterns. 3D box annotation based on data from such sensors is challenging in dynamic scenarios, where objects are observed at different timestamps, hence different positions. Without proper handling of this phenomenon, systematic errors are prone to being introduced in the box annotations. Our work is the first to discover such annotation errors in widely used, publicly available datasets. Through our novel offline estimation method, we correct the annotations so that they follow physically feasible trajectories and achieve spatial and temporal consistency with the sensor data. For the first time, we define metrics for this problem; and we evaluate our method on the Argoverse 2, MAN TruckScenes, and our proprietary datasets. Our approach increases the quality of box annotations by more than 17% in these datasets. Furthermore, we quantify the annotation errors in them and find that the original annotations are misplaced by up to 2.5 m, with highly dynamic objects being the most affected. Finally, we test the impact of the errors in benchmarking and find that the impact is larger than the improvements that state-of-the-art methods typically achieve with respect to the previous state-of-the-art methods; showing that accurate annotations are essential for correct interpretation of performance. Our code is available at https://github.com/alexandre-justo-miro/annotation-correction-3D-boxes.

Problem

• 3D box annotation
• systematic errors
• autonomous driving
• LiDAR
• ground truth

Innovation

• 3D box annotation
• systematic error correction
• temporal consistency
• LiDAR-based perception
• autonomous driving benchmarking