On the Design of Mixture-of-Experts for Dynamic Gaussian Splatting

2026-07-09

AI Summary

Existing dynamic 3D Gaussian splatting methods rely on a single deformation model, which struggles to capture complex real-world motion and consequently suffers from limited robustness. This work introduces the Mixture-of-Experts (MoE) mechanism into dynamic Gaussian splatting for the first time, proposing two multi-deformation modeling paradigms: MoDE jointly optimizes multiple specialized deformation experts within a unified framework, while MoE-GS first optimizes each expert independently and then fuses them via a learnable routing mechanism. Experiments demonstrate that the proposed approaches significantly improve robustness and reconstruction quality in novel view synthesis across diverse dynamic scenes, validating the efficacy of multi-deformation modeling and revealing the critical role of ensemble constraints in shaping expert design and behavior.

Abstract

Dynamic scene reconstruction remains challenging due to the heterogeneous and spatially varying nature of real-world motion. Although recent 3D Gaussian Splatting methods have introduced diverse deformation formulations for dynamic novel view synthesis, each method typically relies on a single deformation model within its representation, which limits robustness across diverse dynamic scenarios. In this work, we study a fundamental problem-multi-deformation modeling for dynamic 3D Gaussian representations-under two distinct integration constraints that differ in when and how multiple deformation experts interact during training. From a Mixture-of-Experts (MoE) perspective, we view multi-deformation modeling as the problem of combining multiple specialized deformation models within a unified 3D representation. We first introduce Mixture of Deformation Experts (MoDE), which integrates multiple deformation experts directly into the deformable Gaussian Splatting pipeline through joint optimization. In MoDE, experts operate on a shared canonical Gaussian representation, enabling multi-deformation modeling without introducing additional training stages or modifying the original optimization schedule. In contrast, we further present Mixture of Experts for Dynamic Gaussian Splatting (MoE-GS) under a different integration constraint, where deformation experts are optimized independently and combined through a separate routing stage. As a result, expert interaction occurs over non-canonical Gaussian representations after individual optimization. Together, these two approaches provide alternative strategies for multi-deformation modeling, clarifying how integration constraints shape the design and behavior of deformation experts in dynamic 3D Gaussian representations. Our code is available at: https://github.com/cvsp-lab/MoE-GS-studio.

Problem

• dynamic scene reconstruction
• multi-deformation modeling
• 3D Gaussian Splatting
• Mixture-of-Experts
• deformation representation

Innovation

• Mixture-of-Experts
• Dynamic Gaussian Splatting
• Multi-deformation Modeling
• Deformable 3D Representation
• Novel View Synthesis