A Late Collaborative Perception Framework for 3D Multi-Object and Multi-Source Association and Fusion

2025-07-03

AI Summary

To address the practical bottlenecks in autonomous driving cooperative perception—namely, high communication bandwidth requirements and the need to share model architectures and parameters (compromising privacy)—this paper proposes a lightweight late-fusion framework that relies solely on shared 3D bounding boxes (i.e., class, position, size, and orientation). To our knowledge, this is the first method enabling accurate cross-heterogeneous-system 3D object fusion without accessing agents’ detection model structures or weights. We design a multi-object association and optimization-based fusion algorithm grounded in geometric and semantic consistency, achieving robust matching and error correction at the fusion layer. Experiments demonstrate substantial improvements: position, scale, and orientation estimation errors are reduced to 1/5, 1/7.5, and 1/2 of baseline levels, respectively. Moreover, the method achieves 100% precision and recall in heterogeneous system fusion, while ensuring computational efficiency, strong generalizability, and strict model privacy preservation.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Search and Optimization: Learning to Search
• Computer Vision: Learning & Optimization for CV

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

In autonomous driving, recent research has increasingly focused on collaborative perception based on deep learning to overcome the limitations of individual perception systems. Although these methods achieve high accuracy, they rely on high communication bandwidth and require unrestricted access to each agent's object detection model architecture and parameters. These constraints pose challenges real-world autonomous driving scenarios, where communication limitations and the need to safeguard proprietary models hinder practical implementation. To address this issue, we introduce a novel late collaborative framework for 3D multi-source and multi-object fusion, which operates solely on shared 3D bounding box attributes-category, size, position, and orientation-without necessitating direct access to detection models. Our framework establishes a new state-of-the-art in late fusion, achieving up to five times lower position error compared to existing methods. Additionally, it reduces scale error by a factor of 7.5 and orientation error by half, all while maintaining perfect 100% precision and recall when fusing detections from heterogeneous perception systems. These results highlight the effectiveness of our approach in addressing real-world collaborative perception challenges, setting a new benchmark for efficient and scalable multi-agent fusion.

Problem

• Overcoming high bandwidth needs in collaborative perception
• Enabling fusion without sharing detection model details
• Improving accuracy in 3D multi-object association

Innovation

• Late fusion using 3D bounding box attributes
• No need for detection model access
• State-of-the-art precision and recall