MoRGS: Efficient Per-Gaussian Motion Reasoning for Streamable Dynamic 3D Scenes

2026-03-26

AI Summary

This work proposes an efficient online 4D reconstruction framework that addresses the common limitation of existing dynamic scene reconstruction methods—namely, their lack of explicit modeling of true 3D motion for Gaussians and susceptibility to pixel-level residuals. For the first time in streaming 3D Gaussian Splatting, the method introduces a per-Gaussian motion inference mechanism. It leverages optical flow from sparse key views as lightweight motion cues to construct a motion offset field and employs a motion-confidence-weighted update strategy to effectively distinguish static and dynamic regions. The approach achieves state-of-the-art performance among online methods, significantly improving reconstruction quality, temporal consistency, and motion fidelity while maintaining real-time efficiency.

Technology Category

• Computer Vision: Motion & Tracking
• Planning, Routing, and Scheduling: Replanning and Plan Repair
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Online reconstruction of dynamic scenes aims to learn from streaming multi-view inputs under low-latency constraints. The fast training and real-time rendering capabilities of 3D Gaussian Splatting have made on-the-fly reconstruction practically feasible, enabling online 4D reconstruction. However, existing online approaches, despite their efficiency and visual quality, fail to learn per-Gaussian motion that reflects true scene dynamics. Without explicit motion cues, appearance and motion are optimized solely under photometric loss, causing per-Gaussian motion to chase pixel residuals rather than true 3D motion. To address this, we propose MoRGS, an efficient online per-Gaussian motion reasoning framework that explicitly models per-Gaussian motion to improve 4D reconstruction quality. Specifically, we leverage optical flow on a sparse set of key views as lightweight motion cues that regularize per-Gaussian motion beyond photometric supervision. To compensate for the sparsity of flow supervision, we learn a per-Gaussian motion offset field that reconciles discrepancies between projected 3D motion and observed flow across views and time. In addition, we introduce a per-Gaussian motion confidence that separates dynamic from static Gaussians and weights Gaussian attribute residual updates, thereby suppressing redundant motion in static regions for better temporal consistency and accelerating the modeling of large motions. Extensive experiments demonstrate that MoRGS achieves state-of-the-art reconstruction quality and motion fidelity among online methods, while maintaining streamable performance.

Problem

• dynamic scene reconstruction
• per-Gaussian motion
• online 4D reconstruction
• motion reasoning
• 3D Gaussian Splatting

Innovation

• per-Gaussian motion
• optical flow regularization
• motion confidence
• online 4D reconstruction
• 3D Gaussian Splatting