🤖 AI Summary
This work addresses the problem of learning stochastic dynamics from endpoint and time-varying marginal distribution observations to generate continuous random trajectories that conform to prescribed marginal laws. The authors reformulate generative modeling as a McKean–Vlasov control problem, replacing hard interpolation or optimal transport constraints with soft energy penalties to enable learning of globally coupled mean-field dynamics. They establish a theoretical connection between the time derivative of the marginal law and a score-like function. Building on a forward–backward stochastic differential equation (FBSDE) framework and leveraging neural SDE solvers, the method accurately reconstructs target marginal distribution trajectories on low-dimensional benchmarks and generates coherent stochastic paths in high-dimensional settings—such as facial images and SMPL-H human motion data—that faithfully evolve according to observed marginal distributions.
📝 Abstract
We propose a generative framework for learning stochastic dynamics from endpoint and intermediate distributional observations. The method formulates generation as a McKean-Vlasov control problem in which terminal and time-marginal laws are enforced through soft energy constraints. The associated optimality system is a forward-backward stochastic differential equation (FBSDE) whose backward component receives a continuous drift induced by the marginal law penalties. This provides a principled alternative to hard interpolation or optimal transport maps between observed distributions: the model learns a stochastic path law whose dynamics remain globally coupled through the mean-field objective. We derive the reduced FBSDE system for quadratic control cost and constant diffusion, connecting terminal and marginal law flat derivatives to score-like training signals. The resulting neural solver is evaluated on low-dimensional distributional benchmarks, where it recovers smooth stochastic paths matching prescribed marginal laws. In a higher-dimensional ALAE latent space, endpoint supervision is used as a qualitative stress test for transporting non-smiling faces toward smiling ones in a pretrained representation. We then use articulated human motion as a structured high-dimensional case study on a curated AMASS low-to-high position dataset, using SMPL-H pose sequences and reduced pose representations. The experiments show that soft marginal law constraints can produce coherent stochastic trajectories whose intermediate distributions follow the observed evolution of human motion. The code is available at https://github.com/murex/deep-mkv-gen/tree/main.