InstanceBEV: Unifying Instance and BEV Representation for Global Modeling

2025-05-20

AI Summary

To address the high computational complexity of multi-view occupancy networks and the engineering-heavy optimization requirements of BEV methods, this paper proposes the first end-to-end Transformer-driven, instance-level BEV global modeling framework. Our approach eliminates sparse representations and custom operators, instead leveraging Transformers directly for full-resolution BEV feature aggregation. Key contributions include: (1) an instance-guided BEV dimensionality compression mechanism that enables semantic-aware, efficient downsampling; (2) a unified pipeline integrating multi-view image encoding, instance-aware 3D BEV sampling, and differentiable occupancy modeling; and (3) a lightweight, plug-and-play architecture requiring no additional engineering tuning. Evaluated on OpenOcc-NuScenes, our method achieves state-of-the-art performance while significantly reducing computational overhead and enhancing BEV semantic density and global modeling efficiency.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multi-instance/Multi-view Learning
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Security and Privacy: Data transparency and provenance
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Occupancy Grid Maps are widely used in navigation for their ability to represent 3D space occupancy. However, existing methods that utilize multi-view cameras to construct Occupancy Networks for perception modeling suffer from cubic growth in data complexity. Adopting a Bird's-Eye View (BEV) perspective offers a more practical solution for autonomous driving, as it provides higher semantic density and mitigates complex object occlusions. Nonetheless, BEV-based approaches still require extensive engineering optimizations to enable efficient large-scale global modeling. To address this challenge, we propose InstanceBEV, the first method to introduce instance-level dimensionality reduction for BEV, enabling global modeling with transformers without relying on sparsification or acceleration operators. Different from other BEV methods, our approach directly employs transformers to aggregate global features. Compared to 3D object detection models, our method samples global feature maps into 3D space. Experiments on OpenOcc-NuScenes dataset show that InstanceBEV achieves state-of-the-art performance while maintaining a simple, efficient framework without requiring additional optimizations.

Problem

• Reducing data complexity in 3D occupancy modeling
• Enabling efficient global BEV modeling with transformers
• Improving semantic density and occlusion handling in autonomous driving

Innovation

• Introduces instance-level dimensionality reduction for BEV
• Uses transformers to aggregate global features directly
• Samples global feature maps into 3D space efficiently