GraphBEV++: Multi-Modal Feature Alignment for Autonomous Driving

2026-06-15

AI Summary

This work addresses the misalignment of Bird’s-Eye-View (BEV) perception features arising from calibration uncertainty between LiDAR and cameras in autonomous driving. To systematically tackle multimodal misalignment induced by projection, the authors propose GraphBEV++, the first framework to explicitly handle this challenge. It integrates LocalAlign-v2 and GlobalAlign-v2 modules for synergistic local and global alignment, is compatible with both LSS- and query-based BEV paradigms, and introduces a novel diffusion-based implicit alignment mechanism that combines graph matching, deformable attention, and cross-modal offset learning. Evaluated on nuScenes, Waymo, and Argoverse2, GraphBEV++ achieves state-of-the-art performance, significantly improving long-range 3D object detection and occupancy prediction accuracy, and consistently outperforms five strong baselines in end-to-end driving tasks.

Abstract

Feature misalignment in BEV perception is a critical yet often overlooked challenge in autonomous driving, especially under calibration uncertainties between LiDAR and camera sensors. To address this issue, we propose a robust multi-modal fusion framework, GraphBEV++, which systematically mitigates projection-induced misalignment. The framework consists of two key modules: LocalAlign-v2 and GlobalAlign-v2. LocalAlign-v2 introduces neighborhood-aware depth features via graph matching to correct local misalignment. It supports both LSS-based and query-based BEV representations, making it compatible with BEVFusion and BEVFormer architectures for consistent cross-paradigm alignment. GlobalAlign-v2 encompasses two variants: Deformable and Diffusion. The Deformable variant addresses global misalignment in LSS-based multi-modal BEV by explicitly learning cross-modal feature offsets. In contrast, the Diffusion variant targets implicit misalignment in query-based BEV by injecting noise to simulate misalignment and employing a denoising process to recover aligned features. Experimental results show that GraphBEV++ achieves state-of-the-art performance under misalignment noise on nuScenes and Waymo subset, improves long-range detection on Argoverse2, and generalizes effectively to the 3D occupancy prediction task, consistently improving occupancy estimation accuracy and robustness under both clean and noisy settings. Furthermore, GraphBEV++ effectively alleviates misalignment issues in end-to-end autonomous driving. Compared with five baselines (UniAD, VAD, FusionAD, MomAD, and WoTE), it demonstrates superior performance in both open-loop (nuScenes) and closed-loop (Bench2Drive and NAVSIM) evaluations across perception, prediction, and planning tasks.

Problem

• feature misalignment
• BEV perception
• multi-modal fusion
• calibration uncertainty
• autonomous driving

Innovation

• multi-modal fusion
• feature alignment
• BEV perception
• graph matching
• calibration uncertainty