DrivingGaussian++: Towards Realistic Reconstruction and Editable Simulation for Surrounding Dynamic Driving Scenes

2025-08-28

AI Summary

To address the challenge of jointly reconstructing and semantically editing dynamic and static elements in autonomous driving scenes, this paper proposes a training-free dynamic scene modeling framework. It employs incremental 3D Gaussian splatting to model the static background, introduces dynamic Gaussians to represent moving objects, and incorporates LiDAR-derived geometric priors to enhance multi-view consistency. Furthermore, it leverages large language models to generate physically plausible object trajectories, enabling semantics-aware scene editing. To our knowledge, this is the first method supporting training-free dynamic scene editing. Quantitative and qualitative evaluations demonstrate superior performance over state-of-the-art approaches in surround-view synthesis quality, geometric fidelity, and editing flexibility—yielding significantly improved reconstruction realism and scene diversity.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Natural Language Processing: Language Grounding & Multi-modal NLP
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Large language models for search
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

We present DrivingGaussian++, an efficient and effective framework for realistic reconstructing and controllable editing of surrounding dynamic autonomous driving scenes. DrivingGaussian++ models the static background using incremental 3D Gaussians and reconstructs moving objects with a composite dynamic Gaussian graph, ensuring accurate positions and occlusions. By integrating a LiDAR prior, it achieves detailed and consistent scene reconstruction, outperforming existing methods in dynamic scene reconstruction and photorealistic surround-view synthesis. DrivingGaussian++ supports training-free controllable editing for dynamic driving scenes, including texture modification, weather simulation, and object manipulation, leveraging multi-view images and depth priors. By integrating large language models (LLMs) and controllable editing, our method can automatically generate dynamic object motion trajectories and enhance their realism during the optimization process. DrivingGaussian++ demonstrates consistent and realistic editing results and generates dynamic multi-view driving scenarios, while significantly enhancing scene diversity. More results and code can be found at the project site: https://xiong-creator.github.io/DrivingGaussian_plus.github.io

Problem

• Realistic reconstruction of dynamic driving scenes
• Controllable editing for autonomous driving simulation
• Accurate modeling of moving objects and occlusions

Innovation

• Incremental 3D Gaussians for static background
• Composite dynamic Gaussian graph for moving objects
• LiDAR prior integration for detailed reconstruction