Post Fusion Bird's Eye View Feature Stabilization for Robust Multimodal 3D Detection

2026-03-05

AI Summary

This work addresses the significant performance degradation of multimodal 3D object detection under domain shifts or sensor failures, which hinders real-world deployment. To enhance robustness without altering existing detector architectures or requiring retraining, the authors propose a lightweight, plug-and-play Post-Fusion Stabilizer (PFS) that operates on the bird’s-eye-view (BEV) intermediate features of pre-trained detectors. PFS adaptively suppresses corrupted regions and recovers weakened cues through a combination of residual correction, spatial masking, and feature statistical stabilization. Evaluated on nuScenes, the method substantially improves detection robustness across various failure scenarios: it boosts mAP by 1.2% under camera failure and by 4.4% in low-light conditions, with only 3.3 million additional parameters.

Technology Category

• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Natural Language Processing: Safety and Robustness

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Camera-LiDAR fusion is widely used in autonomous driving to enable accurate 3D object detection. However, bird's-eye view (BEV) fusion detectors can degrade significantly under domain shift and sensor failures, limiting reliability in real-world deployment. Existing robustness approaches often require modifying the fusion architecture or retraining specialized models, making them difficult to integrate into already deployed systems. We propose a Post Fusion Stabilizer (PFS), a lightweight module that operates on intermediate BEV representations of existing detectors and produces a refined feature map for the original detection head. The design stabilizes feature statistics under domain shift, suppresses spatial regions affected by sensor degradation, and adaptively restores weakened cues through residual correction. Designed as a near-identity transformation, PFS preserves performance while improving robustness under diverse camera and LiDAR corruptions. Evaluations on the nuScenes benchmark demonstrate that PFS achieves state-of-the-art results in several failure modes, notably improving camera dropout robustness by +1.2% and low-light performance by +4.4% mAP while maintaining a lightweight footprint of only 3.3 M parameters.

Problem

• BEV fusion
• domain shift
• sensor failure
• 3D object detection
• robustness

Innovation

• Post Fusion Stabilizer
• Bird's-Eye View
• Multimodal 3D Detection
• Robustness
• Sensor Fusion