Learning Efficient 4D Gaussian Representations from Monocular Videos with Flow Splatting

2026-06-29

AI Summary

Reconstructing dynamic 3D scenes from monocular video remains challenging due to the trade-off between modeling complexity and computational efficiency. Existing 4D Gaussian-based methods often suffer from lengthy training times, slow rendering speeds, or high memory consumption. This work proposes Flow Splatting, which introduces a velocity field into the Gaussian splatting framework for the first time. By leveraging optical flow supervision, the method dynamically learns time-varying Gaussian means and covariances, naturally yielding an approximation of the underlying velocity field. Coupled with an extended volumetric rendering strategy that accounts for camera motion, Flow Splatting achieves state-of-the-art performance across multiple benchmarks, simultaneously improving image quality, training efficiency, and rendering speed.

Abstract

Reconstructing dynamic 3D scenes from monocular videos is challenging due to scene complexity and temporal dynamics. With the advancement of 3D Gaussian Splatting in novel view synthesis, existing methods extend 3D Gaussians to 4D domain with deformation fields, trajectories or spatiotemporal 4D volumes to model scene element deformation. However, these methods suffer from long training time, low rendering speed or high memory consumption for per-frame reconstruction of 4D volumes, without fully exploiting dense dynamic information. To address this issue, we propose Flow Splatting, which constructs the velocity field and enables the conventional splatting technique to render optical flow from the velocity field to supervise dynamics learning process from monocular videos. Specifically, we extend 4D volumes with time varying means and covariance to represent complex dynamics. Then, we construct and approximate the velocity field naturally based on this representations. While conventional volume rendering techniques support to render color fields, we extend the volume rendering strategy to splat the velocity field by considering the influence of camera motions. We conduct experiments on various benchmarks to demonstrate the efficiency and effectiveness of our method. Compared to the state-of-the-art methods, our model achieves better image quality with less time consumption and higher rendering speed.

Problem

• 4D Gaussian Splatting
• monocular video
• dynamic scene reconstruction
• optical flow
• velocity field

Innovation

• Flow Splatting
• 4D Gaussian Splatting
• monocular video
• velocity field
• dynamic scene reconstruction