4DSTR: Advancing Generative 4D Gaussians with Spatial-Temporal Rectification for High-Quality and Consistent 4D Generation
Existing 4D generation methods suffer from limited spatiotemporal consistency and poor modeling of rapid motion, primarily due to the lack of effective joint spatiotemporal representations. To address this, we propose 4DSTR—a generative network based on 4D Gaussian lattices. Our method introduces two key innovations: (1) a spatiotemporal correction mechanism that explicitly optimizes Gaussian ellipsoid scaling and rotation deformations via time-correlation modeling, ensuring temporal coherence; and (2) an adaptive spatial densification and dynamic pruning strategy that responds in real time to geometric changes induced by abrupt motion. By integrating differentiable 4D rendering with learnable Gaussian point insertion and removal, 4DSTR enables end-to-end video-to-4D generation. Evaluated on standard benchmarks, 4DSTR achieves significant improvements in reconstruction accuracy, spatiotemporal continuity, and robustness to fast motion—setting new state-of-the-art performance.