Space-Time Forecasting of Dynamic Scenes with Motion-aware Gaussian Grouping

2026-02-25

AI Summary

This work addresses the challenges of motion incoherence and temporal instability in long-term dynamic scene prediction arising from limited observations. To this end, we propose a physically consistent modeling approach based on 4D Gaussian splatting. By introducing a motion-aware Gaussian grouping mechanism, our method unifies the representation of both rigid and non-rigid motion regions and employs a grouped optimization strategy to construct a spatiotemporally coherent and temporally stable representation. Building upon this foundation, we design a lightweight motion prediction module to enable efficient future scene evolution. Experiments demonstrate that our approach significantly outperforms existing methods on both synthetic and real-world datasets, achieving state-of-the-art performance in rendering quality, motion plausibility, and long-term prediction stability.

Technology Category

• Computer Vision: Motion & Tracking
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Intelligent Robots: Motion and Path Planning

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Forecasting dynamic scenes remains a fundamental challenge in computer vision, as limited observations make it difficult to capture coherent object-level motion and long-term temporal evolution. We present Motion Group-aware Gaussian Forecasting (MoGaF), a framework for long-term scene extrapolation built upon the 4D Gaussian Splatting representation. MoGaF introduces motion-aware Gaussian grouping and group-wise optimization to enforce physically consistent motion across both rigid and non-rigid regions, yielding spatially coherent dynamic representations. Leveraging this structured space-time representation, a lightweight forecasting module predicts future motion, enabling realistic and temporally stable scene evolution. Experiments on synthetic and real-world datasets demonstrate that MoGaF consistently outperforms existing baselines in rendering quality, motion plausibility, and long-term forecasting stability. Our project page is available at https://slime0519.github.io/mogaf

Problem

• dynamic scene forecasting
• motion coherence
• long-term prediction
• space-time representation
• object-level motion

Innovation

• Motion-aware Gaussian Grouping
• 4D Gaussian Splatting
• Dynamic Scene Forecasting
• Group-wise Optimization
• Long-term Temporal Prediction