Ground4D: Consistency-Aware 4D Reconstruction from Monocular Video

2026-06-27

AI Summary

This work addresses the challenge of reconstructing geometrically consistent 4D scene representations from monocular videos to enable dynamic novel view synthesis. The authors propose a two-stage framework: first, leveraging a pre-trained 3D foundation model (e.g., VGGT) to obtain training-free, multi-view consistent geometry and camera poses; second, introducing dynamic Gaussian splatting that incorporates geometric consistency constraints into differentiable rendering to model continuous-time 4D dynamics. This is the first approach to integrate geometric priors from 3D foundation models with dynamic Gaussian splatting, achieving high-quality, training-free initialization and temporally coherent 4D reconstruction across views. The method significantly enhances the fidelity of dynamic novel view synthesis while preserving accurate scene geometry.

Abstract

Learning a 4D scene representation from a single monocular video that supports dynamic novel-view synthesis while maintaining faithful geometry over time remains challenging. Dynamic Gaussian Splatting achieves strong rendering performance through photometric optimization, yet does not explicitly enforce multi-view geometric consistency. In contrast, 3D foundation models recover coherent scene geometry and camera motion, but their point-based outputs are not designed for photorealistic rendering. We propose Ground4D, a geometry-grounded framework built on two stages. First, we perform geometry initialization via 3D foundation models, leveraging VGGT in a training-free manner to reconstruct multi-view-consistent 3D geometry and camera poses from monocular video. The recovered geometry provides a structured and reliable initialization for dynamic Gaussian representations. Second, we conduct geometry-consistency-aware refinement via dynamic Gaussian Splatting, optimizing the representation through differentiable rendering while maintaining multi-view geometric consistency across both observed and synthesized viewpoints. Furthermore, Ground4D inherently models the continuous 4D dynamics of the scene, naturally supporting rendering at arbitrary timestamps. By integrating foundation-level geometric priors into dynamic Gaussian optimization, Ground4D achieves stronger reconstruction fidelity and rendering performance, underscoring the role of geometry-grounded constraints in robust 4D scene modeling.

Problem

• 4D reconstruction
• monocular video
• geometric consistency
• dynamic novel-view synthesis
• scene representation

Innovation

• 4D reconstruction
• geometry consistency
• dynamic Gaussian splatting
• monocular video
• foundation models