L2D2-GS: Learning to Densify for Feedforward Dynamic Gaussian Scene Reconstruction

2026-06-28

AI Summary

This work addresses the high memory overhead of existing feedforward Gaussian reconstruction methods for dynamic scenes at high resolutions and their difficulty in consistently fusing multi-view observations. The authors propose a unified iterative framework that formulates generalizable reconstruction as an alternating process of optimization and densification. Local Gaussian primitives are adaptively densified under the guidance of a self-supervised reward signal, while a reparameterization-based geometric regularization mechanism is introduced to avoid poor local optima. By integrating 3D Gaussian splatting, multi-view feature fusion, and efficient optimization strategies, the method achieves state-of-the-art reconstruction fidelity and strong zero-shot generalization on the PandaSet and Waymo datasets using fewer Gaussian primitives.

Abstract

High-fidelity reconstruction of dynamic urban environments is a cornerstone of autonomous driving simulation and large-scale world modeling. While 3D Gaussian Splatting (3DGS) has established a new standard for real-time rendering, its reliance on expensive per-scene optimization limits scalability. Conversely, recent feedforward methods that infer Gaussian parameters offer faster speed but face fundamental bottlenecks: they are memory-prohibitive at high resolutions and struggle to fuse dense multi-view observations consistently. This paper presents L2D2-GS, a unified framework that reformulates generalizable reconstruction not as a one-shot regression, but as a robust iterative process of optimization and densification. To resolve the ambiguity of supervision in primitive generation, we propose a self-supervised densification policy that derives explicit reward signals from global reconstruction gains to guide local densification. Furthermore, we mitigate irreversible early-stage artifacts through a geometric regularization mechanism, utilizing reparameterization to constrain the optimization manifold and prevent convergence to poor local optima. Extensive experiments on the PandaSet and Waymo datasets demonstrate that our method achieves state-of-the-art reconstruction fidelity and strong zero-shot generalization, while using fewer primitives than competing baselines.

Problem

• dynamic scene reconstruction
• 3D Gaussian Splatting
• feedforward methods
• multi-view fusion
• memory efficiency

Innovation

• 3D Gaussian Splatting
• feedforward reconstruction
• self-supervised densification
• geometric regularization
• zero-shot generalization