DynaFlow: Dynamics-embedded Flow Matching for Physically Consistent Motion Generation from State-only Demonstrations

2025-09-24

AI Summary

This work addresses the problem of generating physically consistent motion trajectories and inferring executable action sequences from state-only demonstrations—i.e., without action labels. To this end, we propose DynaFlow, the first method embedding a differentiable physics simulator into a flow-matching framework to enable end-to-end mapping from observed state sequences to physically feasible trajectories and their corresponding actions. Its core innovation lies in incorporating differentiable dynamics constraints directly into the generative process, ensuring inherent physical plausibility while enabling implicit action inversion and long-horizon open-loop motion generation. By jointly optimizing both the latent state-space trajectory and the generative model, DynaFlow successfully reproduces diverse gaits on the Go1 quadrupedal robot, transforming kinematically infeasible demonstrations into dynamically executable behaviors. Experimental validation on real hardware demonstrates its effectiveness, robustness, and generalization capability across locomotion tasks.

Technology Category

• Intelligent Robots: Motion and Path Planning
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

This paper introduces DynaFlow, a novel framework that embeds a differentiable simulator directly into a flow matching model. By generating trajectories in the action space and mapping them to dynamically feasible state trajectories via the simulator, DynaFlow ensures all outputs are physically consistent by construction. This end-to-end differentiable architecture enables training on state-only demonstrations, allowing the model to simultaneously generate physically consistent state trajectories while inferring the underlying action sequences required to produce them. We demonstrate the effectiveness of our approach through quantitative evaluations and showcase its real-world applicability by deploying the generated actions onto a physical Go1 quadruped robot. The robot successfully reproduces diverse gait present in the dataset, executes long-horizon motions in open-loop control and translates infeasible kinematic demonstrations into dynamically executable, stylistic behaviors. These hardware experiments validate that DynaFlow produces deployable, highly effective motions on real-world hardware from state-only demonstrations, effectively bridging the gap between kinematic data and real-world execution.

Problem

• Generating physically consistent motion from state-only demonstrations
• Inferring underlying action sequences while ensuring dynamic feasibility
• Bridging kinematic data and real-world execution on physical robots

Innovation

• Embeds differentiable simulator into flow matching
• Generates actions mapped to physically feasible states
• Trains on state-only demonstrations with inferred actions