Safety-Aligned 3D Object Detection: Single-Vehicle, Cooperative, and End-to-End Perspectives

2026-04-02

AI Summary

This study addresses the critical gap in existing 3D object detection evaluation and optimization methods, which often overlook safety-critical errors and thus fail to ensure real-world safety in autonomous driving systems. To bridge this gap, the authors propose a safety-aligned evaluation and optimization framework spanning three levels: single-vehicle perception, vehicle-infrastructure cooperative perception, and end-to-end driving. They introduce a safety-oriented metric, NDS-USC, and a novel loss function, EC-IoU, to consistently enhance safety performance across diverse perception paradigms. Experimental results demonstrate that safety-aware fine-tuning significantly improves detection of critical objects, with vehicle-infrastructure cooperation outperforming single-vehicle approaches. Moreover, integrating EC-IoU into the SparseDrive end-to-end framework reduces collision rates by nearly 30%, substantially enhancing system-level safety.

Abstract

Perception plays a central role in connected and autonomous vehicles (CAVs), underpinning not only conventional modular driving stacks, but also cooperative perception systems and recent end-to-end driving models. While deep learning has greatly improved perception performance, its statistical nature makes perfect predictions difficult to attain. Meanwhile, standard training objectives and evaluation benchmarks treat all perception errors equally, even though only a subset is safety-critical. In this paper, we investigate safety-aligned evaluation and optimization for 3D object detection that explicitly characterize high-impact errors. Building on our previously proposed safety-oriented metric, NDS-USC, and safety-aware loss function, EC-IoU, we make three contributions. First, we present an expanded study of single-vehicle 3D object detection models across diverse neural network architectures and sensing modalities, showing that gains under standard metrics such as mAP and NDS may not translate to safety-oriented criteria represented by NDS-USC. With EC-IoU, we reaffirm the benefit of safety-aware fine-tuning for improving safety-critical detection performance. Second, we conduct an ego-centric, safety-oriented evaluation of AV-infrastructure cooperative object detection models, underscoring its superiority over vehicle-only models and demonstrating a safety impact analysis that illustrates the potential contribution of cooperative models to "Vision Zero." Third, we integrate EC-IoU into SparseDrive and show that safety-aware perception hardening can reduce collision rate by nearly 30% and improve system-level safety directly in an end-to-end perception-to-planning framework. Overall, our results indicate that safety-aligned perception evaluation and optimization offer a practical path toward enhancing CAV safety across single-vehicle, cooperative, and end-to-end autonomy settings.

Problem

• safety-aligned perception
• 3D object detection
• safety-critical errors
• autonomous vehicles
• perception evaluation

Innovation

• safety-aligned perception
• 3D object detection
• cooperative perception
• end-to-end driving
• EC-IoU