Variational Inference for Bird's Eye View Segmentation in Autonomous Driving

2026-07-16

AI Summary

This work addresses the challenges of multi-camera data fusion and insufficient bird’s-eye-view (BEV) segmentation accuracy in complex driving scenarios by proposing a Transformer-based Variational Bifurcated network (TVB). TVB is the first approach to integrate variational inference with normalizing flows for BEV segmentation. It implicitly learns the mapping from multi-view images to a unified BEV representation through posterior BEV supervision, generating multiple candidate maps. A novel BEV-attention fusion module adaptively aggregates these candidates to enhance the realism and expressiveness of the resulting map. Evaluated on the nuScenes and OPV2V datasets, the proposed method significantly outperforms existing approaches in both multi-camera BEV segmentation and lane perception tasks, achieving state-of-the-art performance.

Abstract

The bird's eye view (BEV) has emerged as a pivotal approach for environmental perception in autonomous driving, providing a unified spatial representation for vehicles. Nevertheless, despite BEV's significance in addressing the challenges inherent to autonomous driving, effectively fusing data from multiple camera sensors and operating in complex external driving environments remains a considerable challenge. To mitigate this issue, we recast the BEV segmentation problem within a variational inference framework. In this paper, we propose a novel transformer-based variational flow transformation network for BEV segmentation, denoted as TVB. Our architecture implicitly learns the mapping from multiple camera views to a unified canonical BEV map during training by exploiting posterior BEV supervision. TVB employs a conditional variational auto encoder (CVAE) as its backbone and produces multiple BEV map candidates. To augment the realism of the generated BEV maps, we integrate normalizing flows into the map generation process, enabling the construction of more complex and expressive probability distributions. Furthermore, we design a BEV-attention fusion (BAF) module that harnesses attention mechanisms to adaptively integrate the multiple candidate BEV maps. Experimental results, evaluated on both the nuScenes and OPV2Vdatasets, demonstrate that our proposed method achieves superior performance in multi-camera view BEV segmentation and lane environment perception.

Problem

• Bird's Eye View Segmentation
• Multi-camera Fusion
• Autonomous Driving
• Environmental Perception

Innovation

• Variational Inference
• Bird's Eye View Segmentation
• Normalizing Flows
• Conditional Variational Autoencoder
• BEV-Attention Fusion