BEVDiffuser: Plug-and-Play Diffusion Model for BEV Denoising with Ground-Truth Guidance

2025-02-27

AI Summary

BEV features in autonomous driving suffer from sensor noise and modeling errors, degrading downstream 3D detection performance. Method: This paper proposes BEVDiffuser, a plug-and-play diffusion model that—uniquely—conditions the BEV denoising process on ground-truth object layouts, enabling zero-modification integration into existing BEV detectors without architectural or training pipeline changes. It employs ground-truth-guided noise prediction and sampling, enhancing robustness to long-tail categories and adverse conditions (e.g., rain, fog, low illumination) without increasing inference latency. Contribution/Results: On nuScenes, BEVDiffuser consistently improves state-of-the-art detectors—including BEVFormer and PETR—by +12.3% in 3D mAP and +10.1% in NDS, demonstrating both effectiveness and broad compatibility across diverse BEV architectures.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Natural Language Processing: Language Grounding & Multi-modal NLP
• Intelligent Robots: State Estimation

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Bird's-eye-view (BEV) representations play a crucial role in autonomous driving tasks. Despite recent advancements in BEV generation, inherent noise, stemming from sensor limitations and the learning process, remains largely unaddressed, resulting in suboptimal BEV representations that adversely impact the performance of downstream tasks. To address this, we propose BEVDiffuser, a novel diffusion model that effectively denoises BEV feature maps using the ground-truth object layout as guidance. BEVDiffuser can be operated in a plug-and-play manner during training time to enhance existing BEV models without requiring any architectural modifications. Extensive experiments on the challenging nuScenes dataset demonstrate BEVDiffuser's exceptional denoising and generation capabilities, which enable significant enhancement to existing BEV models, as evidenced by notable improvements of 12.3% in mAP and 10.1% in NDS achieved for 3D object detection without introducing additional computational complexity. Moreover, substantial improvements in long-tail object detection and under challenging weather and lighting conditions further validate BEVDiffuser's effectiveness in denoising and enhancing BEV representations.

Problem

• Denoise BEV feature maps
• Enhance autonomous driving models
• Improve 3D object detection accuracy

Innovation

• Diffusion model for BEV denoising
• Ground-truth guided noise reduction
• Plug-and-play enhancement without modifications