Telegrapher's Generative Model via Kac Flows

2025-06-25

AI Summary

Conventional flow-based generative models suffer from insufficient Lipschitz continuity of the probability flow and non-global boundedness of the velocity norm, leading to training instability and poor scalability. Method: This paper introduces, for the first time, the damped wave equation (telegraph equation) coupled with the Kac random process into generative modeling. By establishing a theoretical correspondence between multidimensional Kac processes and absolutely continuous curves in Wasserstein space, we derive analytical constraints on the velocity field that ensure both Lipschitz continuity and global boundedness of the velocity norm. The resulting dynamics are trained within the flow matching framework, where neural networks approximate the conditional velocity field to enable efficient sampling. Contribution/Results: The proposed model achieves enhanced stability, asymptotic consistency, and high-dimensional scalability. Experiments demonstrate superior generation quality over diffusion models on high-dimensional data, improved training stability, and significantly reduced velocity fluctuations.

Technology Category

• Machine Learning: Deep Generative Models & Autoencoders
• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Natural Language Processing: Generation

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

We break the mold in flow-based generative modeling by proposing a new model based on the damped wave equation, also known as telegrapher's equation. Similar to the diffusion equation and Brownian motion, there is a Feynman-Kac type relation between the telegrapher's equation and the stochastic Kac process in 1D. The Kac flow evolves stepwise linearly in time, so that the probability flow is Lipschitz continuous in the Wasserstein distance and, in contrast to diffusion flows, the norm of the velocity is globally bounded. Furthermore, the Kac model has the diffusion model as its asymptotic limit. We extend these considerations to a multi-dimensional stochastic process which consists of independent 1D Kac processes in each spatial component. We show that this process gives rise to an absolutely continuous curve in the Wasserstein space and compute the conditional velocity field starting in a Dirac point analytically. Using the framework of flow matching, we train a neural network that approximates the velocity field and use it for sample generation. Our numerical experiments demonstrate the scalability of our approach, and show its advantages over diffusion models.

Problem

• Proposes new generative model using telegrapher's equation
• Addresses limitations of diffusion models with Kac flows
• Trains neural network for scalable sample generation

Innovation

• New model based on telegrapher's equation
• Kac flow with Lipschitz continuous probability
• Neural network approximates velocity field