Sparse4D-Radar: An Efficient and Robust Framework for Surround-View 3D Object Detection via 4D Radar-Camera Fusion

2026-07-04

AI Summary

This work addresses the limitations of existing 4D radar–camera fusion methods, which are predominantly confined to front-view perception and struggle to achieve efficient and robust surround-view 3D object detection. To overcome this, we propose Sparse4D-Radar, a lightweight surround-view fusion framework that jointly models multimodal features through sparse query representations. The approach introduces a deformable fusion module, Velocity-Consistent Sampling (VCS), and Adaptive Modality Gating (AMG) to dynamically optimize features and adjust modality weights. Evaluated on the OmniHD-Scenes dataset, Sparse4D-Radar achieves state-of-the-art performance, improving mean Average Precision (mAP) by over 7% and Overall Detection Score (ODS) by more than 10% in complex scenarios, while maintaining an inference speed close to 10 FPS—significantly enhancing localization accuracy and modality robustness.

Abstract

In recent years, 4D imaging radar has gained wide attention in autonomous driving for its robustness against harsh weather and ability to output target velocity. Nevertheless, mainstream 4D radar-camera fusion methods only support front-view perception, lacking mature solutions for surround-view sensing. Directly expanding these pipelines to full 360° coverage introduces excessive computation cost and limits real-world deployment. To tackle these limitations, this work proposes Sparse4D-Radar, an efficient robust surround-view multi-modal fusion framework. We first design a Deformable Fusion module to embed radar-camera features into sparse queries, constructing the lightweight base version Sparse4D-Radar-Base. Two dedicated modules are further introduced to boost localization accuracy and modality stability: Velocity-Consistency Sampling (VCS) refines features via radar velocity cues for motion awareness, and Adaptive Modality Gating (AMG) dynamically adjusts cross-modal fusion weights according to feature confidence. Combining all components, we build Sparse4D-Radar-Acc for high-precision detection demands. Comprehensive experiments on OmniHD-Scenes verify that our approach achieves state-of-the-art surround-view 3D detection performance. Compared with prior arts, our method obtains over 7% mAP and 10% ODS improvements under complex driving scenes while running at nearly 10 FPS, striking a favorable trade-off among detection accuracy, environmental robustness and inference efficiency. Our open-source code is available at https://github.com/Aiuan/Sparse4D-Radar.

Problem

• 4D radar
• camera fusion
• surround-view
• 3D object detection
• autonomous driving

Innovation

• 4D radar-camera fusion
• surround-view 3D object detection
• sparse query representation
• velocity-consistency sampling
• adaptive modality gating