Neural Hamiltonian Deformation Fields for Dynamic Scene Rendering

2025-12-11

AI Summary

To address unphysical motion and geometric distortion in dynamic novel view synthesis, this work proposes a Hamiltonian mechanics–based neural deformation field that models the spatiotemporal evolution of Gaussian ellipsoids as energy-conserving trajectories in phase space. We introduce Hamiltonian neural networks to dynamic Gaussian modeling for the first time; design a Boltzmann equilibrium decomposition mechanism for adaptive separation of static and dynamic Gaussians; and incorporate second-order symplectic integration with local rigidity regularization to capture realistic dissipative dynamics. The method integrates scale-aware mipmapping and progressive optimization, enabling physically plausible, high-fidelity, and real-time streamable dynamic rendering across diverse complex motion scenes. Experiments demonstrate significant improvements in the joint optimization of motion naturalness and rendering efficiency.

Technology Category

• Computer Vision: Motion & Tracking
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Intelligent Robots: Motion and Path Planning

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Representing and rendering dynamic scenes with complex motions remains challenging in computer vision and graphics. Recent dynamic view synthesis methods achieve high-quality rendering but often produce physically implausible motions. We introduce NeHaD, a neural deformation field for dynamic Gaussian Splatting governed by Hamiltonian mechanics. Our key observation is that existing methods using MLPs to predict deformation fields introduce inevitable biases, resulting in unnatural dynamics. By incorporating physics priors, we achieve robust and realistic dynamic scene rendering. Hamiltonian mechanics provides an ideal framework for modeling Gaussian deformation fields due to their shared phase-space structure, where primitives evolve along energy-conserving trajectories. We employ Hamiltonian neural networks to implicitly learn underlying physical laws governing deformation. Meanwhile, we introduce Boltzmann equilibrium decomposition, an energy-aware mechanism that adaptively separates static and dynamic Gaussians based on their spatial-temporal energy states for flexible rendering. To handle real-world dissipation, we employ second-order symplectic integration and local rigidity regularization as physics-informed constraints for robust dynamics modeling. Additionally, we extend NeHaD to adaptive streaming through scale-aware mipmapping and progressive optimization. Extensive experiments demonstrate that NeHaD achieves physically plausible results with a rendering quality-efficiency trade-off. To our knowledge, this is the first exploration leveraging Hamiltonian mechanics for neural Gaussian deformation, enabling physically realistic dynamic scene rendering with streaming capabilities.

Problem

• Modeling dynamic scenes with physically plausible motions
• Incorporating Hamiltonian mechanics for realistic deformation fields
• Separating static and dynamic elements via energy-aware mechanisms

Innovation

• Neural deformation field governed by Hamiltonian mechanics
• Energy-aware Boltzmann equilibrium decomposition for static-dynamic separation
• Physics-informed constraints with symplectic integration and rigidity regularization