NEXUS: Neural Energy Fields for Physically Consistent Contact-Rich 3D Object Dynamics

2026-06-12

AI Summary

Existing approaches struggle to jointly model conservative (e.g., gravity, elasticity) and non-conservative (e.g., damping, collisional dissipation) physical effects in complex 3D scenes involving contact, deformation, and external forces, often resulting in dynamics that lack physical consistency. This work proposes NEXUS, a novel framework that unifies neural energy fields with Rayleigh-type dissipation for the first time. By representing objects and their dynamic contact relationships through a graph structure, NEXUS implicitly defines forces via differentiable energy and dissipation functions and evolves trajectories using a multi-substep semi-implicit integrator. The method enables composable, end-to-end learning of physically consistent motion generation, significantly outperforming both learned and physics-structured baselines in trajectory prediction and contact-intensive video synthesis across diverse combinations of material properties and physical effects.

Abstract

Physics-grounded video generation requires controllable 3D object dynamics that remain physically consistent under contact, deformation, and external forcing. Existing trajectory-based methods often model isolated physical effects, making it difficult to compose conservative and non-conservative dynamics in contact-rich 3D scenes. We present NEXUS, a neural energy-field framework for contact-rich 3D object dynamics. NEXUS represents each object as a structural graph and constructs dynamic object-object and object-environment contact graphs. Inspired by Hamiltonian Neural Networks, NEXUS formulates motion through scalar energy and dissipation terms rather than directly predicting states or accelerations. Conservative effects, including gravity and elastic deformation, are composed as additive energy terms, while non-conservative effects such as damping and impact-induced energy loss are modeled with learned Rayleigh-style dissipation. Forces are derived by differentiating the energy and dissipation functions and rolled out with a multi-substep semi-implicit integrator. Across controlled trajectory benchmarks, NEXUS improves long-horizon accuracy over representative learned and physics-structured dynamics baselines under varying mechanical properties and physical-effect compositions. We further show that NEXUS trajectories provide effective guidance for contact-rich video generation, improving physical plausibility while maintaining competitive visual quality.

Problem

• physically consistent dynamics
• contact-rich 3D scenes
• conservative and non-conservative forces
• 3D object dynamics
• physics-grounded video generation

Innovation

• Neural Energy Fields
• Physically Consistent Dynamics
• Contact-Rich Interaction
• Hamiltonian Neural Networks
• Rayleigh Dissipation