NewtonGS: Physics-Structured Object-Level Neural Newtonian Dynamics for Gaussian Scene Animation

2026-08-06

AI Summary

Existing static 3D Gaussian scene representations lack object-level dynamic state modeling, hindering controllable animation. This work proposes NewtonGS, a novel framework that embeds Newtonian mechanical structure into Gaussian scenes for the first time. It models object motion using a compact 22-dimensional state vector encoding pose, velocity, scale changes, and contact properties, and introduces a discrete event mapping to handle ground contact. By integrating analytical dynamics with learned residuals, the method formulates Gaussian Neural Newtonian Dynamics (Gaussian-NND) to jointly update the means and covariances of associated Gaussians. Evaluated on the State-32 and Gaussian-32 datasets, NewtonGS significantly outperforms five analytical baselines in trajectory RMSE, final displacement error, and velocity RMSE, demonstrating its effectiveness in dynamic state prediction and controllable animation synthesis.

Abstract

Animating objects in a static 3D Gaussian scene requires an explicit object-level dynamic state and a controllable model of object motion. Existing dynamic Gaussian methods primarily reconstruct time-varying scenes or simulate deformation, rather than provide compact object states for direct control. To address this gap, we present NewtonGS, a physics-structured framework for object-level state rollout and Gaussian scene animation. NewtonGS represents each object with a 22-dimensional state covering pose, linear and angular velocity, anisotropic scale and its rate, mass, and contact properties. Its Gaussian Neural Newtonian Dynamics (Gaussian-NND) model combines analytic translation, quaternion kinematics, gravity, damping, and scale-restoration dynamics with learned continuous and contact residuals. A discrete event map handles floor contact. Predicted poses and scales define a shared affine transformation that updates the means and covariances of all Gaussians associated with each object. We construct two procedurally generated datasets: State-32 for state-rollout evaluation and Gaussian-32 for state-to-Gaussian transformation. On both the in-distribution and velocity-range-shift splits of State-32, NewtonGS achieves lower trajectory RMSE, final displacement error, and velocity RMSE than five analytic baselines. Experiments on Gaussian-32 further demonstrate effective conversion from predicted states to animated Gaussian objects.

Problem

• Gaussian scene animation
• object-level dynamics
• dynamic state representation
• controllable motion
• 3D object animation

Innovation

• Gaussian Splatting
• Neural Dynamics
• Physics-Based Animation
• Object-Level State
• Newtonian Mechanics