MAPo : Motion-Aware Partitioning of Deformable 3D Gaussian Splatting for High-Fidelity Dynamic Scene Reconstruction

2025-08-27

AI Summary

Existing deformable 3D Gaussian splatting methods suffer from motion blur and loss of fine dynamic details in highly dynamic regions due to uniform spatiotemporal modeling. To address this, we propose a motion-aware deformable 3D Gaussian lattice segmentation framework. Our approach first estimates motion intensity via a dynamic score to guide spatiotemporal adaptive partitioning. Second, it recursively subdivides time intervals in high-motion regions and models each segment independently. Third, it introduces a cross-frame consistency loss to ensure visual coherence across frames. Leveraging multiple network replicas, the framework enables localized, high-fidelity modeling while maintaining computational efficiency comparable to baseline methods. Experiments demonstrate significant improvements in reconstruction accuracy and rendering quality for fast and complex motions, achieving state-of-the-art performance across multiple dynamic scene benchmarks.

Technology Category

• Computer Vision: Motion & Tracking
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Intelligent Robots: Motion and Path Planning

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

3D Gaussian Splatting, known for enabling high-quality static scene reconstruction with fast rendering, is increasingly being applied to dynamic scene reconstruction. A common strategy involves learning a deformation field to model the temporal changes of a canonical set of 3D Gaussians. However, these deformation-based methods often produce blurred renderings and lose fine motion details in highly dynamic regions due to the inherent limitations of a single, unified model in representing diverse motion patterns. To address these challenges, we introduce Motion-Aware Partitioning of Deformable 3D Gaussian Splatting (MAPo), a novel framework for high-fidelity dynamic scene reconstruction. Its core is a dynamic score-based partitioning strategy that distinguishes between high- and low-dynamic 3D Gaussians. For high-dynamic 3D Gaussians, we recursively partition them temporally and duplicate their deformation networks for each new temporal segment, enabling specialized modeling to capture intricate motion details. Concurrently, low-dynamic 3DGs are treated as static to reduce computational costs. However, this temporal partitioning strategy for high-dynamic 3DGs can introduce visual discontinuities across frames at the partition boundaries. To address this, we introduce a cross-frame consistency loss, which not only ensures visual continuity but also further enhances rendering quality. Extensive experiments demonstrate that MAPo achieves superior rendering quality compared to baselines while maintaining comparable computational costs, particularly in regions with complex or rapid motions.

Problem

• Enhances dynamic scene reconstruction fidelity using 3D Gaussian Splatting
• Addresses motion blur in highly dynamic regions via temporal partitioning
• Resolves visual discontinuities at partition boundaries with consistency loss

Innovation

• Motion-aware partitioning strategy for dynamic Gaussians
• Recursive temporal segmentation with duplicated deformation networks
• Cross-frame consistency loss ensuring visual continuity