Beyond Perfect Priors: Adaptive Gaussian Graph for 4D Driving Reconstruction in the Wild

2026-07-07

AI Summary

Existing 4D driving scene reconstruction methods rely on precise priors—such as camera poses, LiDAR depth, or manual annotations—and are prone to optimization ambiguities and topological errors under the noisy conditions typical of in-the-wild videos. This work proposes a self-correcting Adaptive Gaussian Graph (AGG) framework that decouples the optimization of static backgrounds and dynamic objects via a semantics-guided Tick-Tock strategy. AGG further incorporates an adaptive topology evolution module that dynamically refines the graph structure through proxy generation, Gaussian reclassification, and false detection removal. To our knowledge, AGG is the first method to achieve robust 4D reconstruction in unconstrained outdoor scenes without requiring accurate priors, significantly outperforming existing approaches on both KITTI and the newly introduced Wild-30 benchmark while delivering higher visual fidelity and strong tolerance to noisy priors.

Abstract

Reconstructing 4D driving scenes in the wild (e.g., internet and AI-generated videos) is critical for diverse autonomous driving simulation. While recent Gaussian Scene Graph (GSG) methods achieve impressive visual quality, they heavily rely on precise priors, such as accurate camera poses and LiDAR depth, or manual annotations. When initialized with noisy priors estimated from in-the-wild videos, existing GSG methods suffer from optimization ambiguity (e.g., entangling camera and agent poses) and topological failures (e.g., missing objects), causing severe rendering artifacts. To enable robust in-the-wild reconstruction, we introduce Adaptive Gaussian Graph (AGG), a self-correcting 4D framework. Our Semantically-Guided Tick-Tock Strategy leverages 2D foundation features to explicitly decouple static background and camera pose updates from dynamic agent learning. Concurrently, our Adaptive Topology Evolution module actively rectifies graph structures by spawning missing agents, reassigning misclassified Gaussians, and pruning false positives. To rigorously evaluate this in-the-wild setting, we introduce Wild-30, a challenging benchmark of internet and generative videos. Extensive experiments on KITTI and Wild-30 validate that AGG consistently outperforms state-of-the-art approaches in visual fidelity and robustness under noisy priors.

Problem

• 4D driving reconstruction
• in-the-wild videos
• noisy priors
• optimization ambiguity
• topological failures

Innovation

• Adaptive Gaussian Graph
• 4D scene reconstruction
• in-the-wild video
• semantic-guided optimization
• topology evolution