RAF: Reliability-Aware Fusion of Camera, LiDAR, and 4D RADAR for Robust 3D Object Detection in Adverse Weather

2026-07-05

AI Summary

This work addresses the significant performance degradation of 3D object detection under adverse weather conditions, where LiDAR and 4D RADAR point clouds become sparse and camera images suffer from occlusions. To tackle this challenge, the authors propose a Reliability-Aware Fusion (RAF) framework that introduces a camera branch on top of a frozen pre-trained LiDAR-RADAR backbone and, for the first time, explicitly supervises pixel-level visual reliability through learning. This enables dynamic suppression of unreliable image information for robust multimodal fusion. The approach integrates a BEV fusion encoder with a jointly trained, learnable reliability map. Evaluated on the K-Radar and VoD datasets, RAF substantially outperforms LiDAR-RADAR baselines, achieving gains of up to +6.5 AP_BEV and +7.4 AP_3D.

Abstract

Robust 3D object detection in adverse weather conditions is challenging due to sensor limitations. Although combining complementary modalities such as LiDAR and 4D RADAR has shown promise, the sparsity of these sensors becomes apparent in adverse weather with reduced reflections, leading to objects with few or no point cloud returns. To address this limitation, camera sensors provide visual cues even when LiDAR and RADAR signals are weakened. However, cameras themselves are also vulnerable to adverse weather, where some regions become unreliable due to snow or rain occluding the camera lens. While some camera-fusion methods designed for adverse weather learn to weigh image regions via confidence maps, these maps receive no direct supervision and are learned solely through the detection loss. We introduce Reliability-Aware Fusion (RAF), which explicitly supervises per-pixel reliability estimation and provides a direct learning signal for identifying and suppressing unreliable visual cues. Our framework leverages pretrained LiDAR-RADAR networks, keeping their backbones frozen while only training the added camera branch, BEV fusion encoder, and detection head. Extensive experiments on the K-Radar and VoD datasets demonstrate that integrating RAF consistently improves detection accuracy over LiDAR-RADAR baselines, achieving up to +6.5 $AP_{BEV}$ and +7.4 $AP_{3D}$ gains. Code is available at https://github.com/parkie0517/RAF.

Problem

• 3D object detection
• adverse weather
• sensor fusion
• camera reliability
• LiDAR-RADAR

Innovation

• Reliability-Aware Fusion
• 3D Object Detection
• Adverse Weather Perception
• Multi-sensor Fusion
• Camera-LiDAR-RADAR Integration