Neural Robot Dynamics

2025-08-21

AI Summary

Existing neural simulators suffer from poor generalization, struggling to transfer to novel tasks or environments—primarily due to inadequate global state representation and the absence of explicit contact dynamics modeling. This work introduces NeRD, a robot-centric, spatially invariant neural dynamics modeling framework that centers on joint rigid-body state evolution and explicitly encodes contact constraints. NeRD integrates robotic priors into an end-to-end trainable neural solver, designed as a plug-and-play backend compatible with mainstream simulation platforms. Crucially, NeRD is the first neural simulator enabling fine-tuning directly on real-robot data, thereby achieving cross-configuration and cross-environment generalization. Experiments demonstrate high accuracy and numerical stability over thousand-step simulations, support policy learning with purely neural engines, and successfully enable sim-to-real domain adaptation.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Search and Retrieval-Augmented AI: Personalized, context-aware and across-device search
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Accurate and efficient simulation of modern robots remains challenging due to their high degrees of freedom and intricate mechanisms. Neural simulators have emerged as a promising alternative to traditional analytical simulators, capable of efficiently predicting complex dynamics and adapting to real-world data; however, existing neural simulators typically require application-specific training and fail to generalize to novel tasks and/or environments, primarily due to inadequate representations of the global state. In this work, we address the problem of learning generalizable neural simulators for robots that are structured as articulated rigid bodies. We propose NeRD (Neural Robot Dynamics), learned robot-specific dynamics models for predicting future states for articulated rigid bodies under contact constraints. NeRD uniquely replaces the low-level dynamics and contact solvers in an analytical simulator and employs a robot-centric and spatially-invariant simulation state representation. We integrate the learned NeRD models as an interchangeable backend solver within a state-of-the-art robotics simulator. We conduct extensive experiments to show that the NeRD simulators are stable and accurate over a thousand simulation steps; generalize across tasks and environment configurations; enable policy learning exclusively in a neural engine; and, unlike most classical simulators, can be fine-tuned from real-world data to bridge the gap between simulation and reality.

Problem

• Learning generalizable neural simulators for articulated rigid robots
• Addressing inadequate global state representation in existing neural simulators
• Replacing low-level dynamics and contact solvers in analytical simulators

Innovation

• Neural Robot Dynamics (NeRD) model
• Replaces low-level analytical solvers
• Robot-centric spatially-invariant state representation