RayFusion: Ray Fusion Enhanced Collaborative Visual Perception

2025-10-09

AI Summary

To address the ambiguity in 3D object detection caused by missing depth information in vision-only collaborative perception, this paper proposes a ray-fusion-based collaborative visual perception method. The core innovation is the first introduction of ray-wise occupancy modeling, which leverages camera geometric priors to align and fuse ray-level occupancy predictions across multiple vehicle views, effectively suppressing redundant responses and false positives along the line of sight. By integrating differentiable ray sampling with occupancy modeling, the method enhances depth perception robustly—without requiring depth supervision or auxiliary sensors. Extensive experiments demonstrate state-of-the-art performance on major collaborative perception benchmarks, including DAIR-V2X and V2XSet, achieving absolute gains of 8.2–12.6% in 3D detection mAP. The source code is publicly available.

Technology Category

• Computer Vision: 3D Computer Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Collaborative visual perception methods have gained widespread attention in the autonomous driving community in recent years due to their ability to address sensor limitation problems. However, the absence of explicit depth information often makes it difficult for camera-based perception systems, e.g., 3D object detection, to generate accurate predictions. To alleviate the ambiguity in depth estimation, we propose RayFusion, a ray-based fusion method for collaborative visual perception. Using ray occupancy information from collaborators, RayFusion reduces redundancy and false positive predictions along camera rays, enhancing the detection performance of purely camera-based collaborative perception systems. Comprehensive experiments show that our method consistently outperforms existing state-of-the-art models, substantially advancing the performance of collaborative visual perception. The code is available at https://github.com/wangsh0111/RayFusion.

Problem

• Addresses depth ambiguity in camera-based perception systems
• Reduces redundancy and false positives in collaborative detection
• Enhances 3D object detection performance using ray occupancy information

Innovation

• Ray-based fusion method for collaborative perception
• Uses ray occupancy data from collaborators
• Reduces redundancy and false positive predictions