OG-Gaussian: Occupancy Based Street Gaussians for Autonomous Driving

2025-02-20

AI Summary

To address the high cost of LiDAR reliance and manual annotation in autonomous driving simulation, this paper proposes a purely vision-based, unsupervised method for dynamic 3D street scene reconstruction. Our approach eliminates the need for LiDAR data or pre-annotated supervision, operating solely on surround-view camera images. The core contributions are: (1) an Occupancy Prediction Network (ONet) that generates semantic-aware voxel grids—replacing LiDAR point clouds—as geometric priors to guide 3D Gaussian reconstruction; and (2) a dual-stream 3D Gaussian Splatting framework that jointly optimizes static scene rendering and dynamic vehicle pose/trajectory estimation, enabling end-to-end co-modeling of static and dynamic elements. Evaluated on the Waymo Open Dataset, our method achieves 35.13 PSNR and 143 FPS rendering speed—matching state-of-the-art performance—while substantially reducing computational and hardware costs.

Technology Category

• Computer Vision: 3D Computer Vision
• Search and Optimization: Sampling/Simulation-based Search
• Planning, Routing, and Scheduling: Replanning and Plan Repair

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Accurate and realistic 3D scene reconstruction enables the lifelike creation of autonomous driving simulation environments. With advancements in 3D Gaussian Splatting (3DGS), previous studies have applied it to reconstruct complex dynamic driving scenes. These methods typically require expensive LiDAR sensors and pre-annotated datasets of dynamic objects. To address these challenges, we propose OG-Gaussian, a novel approach that replaces LiDAR point clouds with Occupancy Grids (OGs) generated from surround-view camera images using Occupancy Prediction Network (ONet). Our method leverages the semantic information in OGs to separate dynamic vehicles from static street background, converting these grids into two distinct sets of initial point clouds for reconstructing both static and dynamic objects. Additionally, we estimate the trajectories and poses of dynamic objects through a learning-based approach, eliminating the need for complex manual annotations. Experiments on Waymo Open dataset demonstrate that OG-Gaussian is on par with the current state-of-the-art in terms of reconstruction quality and rendering speed, achieving an average PSNR of 35.13 and a rendering speed of 143 FPS, while significantly reducing computational costs and economic overhead.

Problem

• Reconstruct 3D scenes without LiDAR
• Separate dynamic and static objects
• Estimate object trajectories without annotations

Innovation

• Occupancy Grids replace LiDAR
• Learning-based trajectory estimation
• Semantic separation of dynamic objects