WeightFlow: Learning Stochastic Dynamics via Evolving Weight of Neural Network

2025-08-01

AI Summary

Modeling stochastic dynamical systems from discrete observations remains challenging due to discontinuous probability density evolution estimation and the curse of dimensionality in high-dimensional settings. Method: This paper introduces a novel “distribution-to-weight mapping” paradigm: it projects the probability flow in measure space onto the weight space of neural networks and constructs a weight dynamics model governed by graph-controlled differential equations. Crucially, it establishes, for the first time, a theoretical connection between dynamic optimal transport in measure space and an energy functional in weight space, enabling differentiable and scalable parameterization of distributional evolution. The method integrates graph neural networks with dynamic optimal transport theory. Contribution/Results: Evaluated on multi-disciplinary benchmark datasets, the approach achieves an average 43.02% performance improvement over state-of-the-art methods, significantly enhancing both modeling accuracy and scalability for high-dimensional stochastic dynamical systems.

Technology Category

• Machine Learning: Probabilistic Circuits and Graphical Models
• Reasoning under Uncertainty: Stochastic Optimization
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Web Mining and Content Analysis: Models for Web evolution
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Modeling stochastic dynamics from discrete observations is a key interdisciplinary challenge. Existing methods often fail to estimate the continuous evolution of probability densities from trajectories or face the curse of dimensionality. To address these limitations, we presents a novel paradigm: modeling dynamics directly in the weight space of a neural network by projecting the evolving probability distribution. We first theoretically establish the connection between dynamic optimal transport in measure space and an equivalent energy functional in weight space. Subsequently, we design WeightFlow, which constructs the neural network weights into a graph and learns its evolution via a graph controlled differential equation. Experiments on interdisciplinary datasets demonstrate that WeightFlow improves performance by an average of 43.02% over state-of-the-art methods, providing an effective and scalable solution for modeling high-dimensional stochastic dynamics.

Problem

• Modeling stochastic dynamics from discrete observations
• Overcoming curse of dimensionality in density estimation
• Learning neural network weight evolution for dynamics

Innovation

• Model dynamics in neural network weight space
• Connect dynamic transport to energy functional
• Learn evolution via graph differential equation