Learning Latent Energy-Based Models via Interacting Particle Langevin Dynamics

2025-10-14

AI Summary

This work addresses learning latent variable models with energy-based priors. Methodologically, it formulates maximum marginal likelihood estimation as an analytically tractable stochastic differential equation system governed by interacting particle Langevin dynamics, wherein inter-particle interactions jointly evolve the latent variables and prior distribution; a provably convergent discretization algorithm is further designed. The key contribution is the first principled integration of particle-based sampling with energy-based modeling, yielding the first continuous-time learning paradigm that simultaneously offers theoretical convergence guarantees and computational feasibility. Experiments demonstrate stable convergence and high-fidelity sample generation on both synthetic data and image synthesis tasks, consistently outperforming conventional gradient-based and MCMC-based baselines.

Technology Category

• Machine Learning: Probabilistic Circuits and Graphical Models
• Reasoning under Uncertainty: Relational Probabilistic Models
• Computer Vision: Diffusion Models for Vision

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

We develop interacting particle algorithms for learning latent variable models with energy-based priors. To do so, we leverage recent developments in particle-based methods for solving maximum marginal likelihood estimation (MMLE) problems. Specifically, we provide a continuous-time framework for learning latent energy-based models, by defining stochastic differential equations (SDEs) that provably solve the MMLE problem. We obtain a practical algorithm as a discretisation of these SDEs and provide theoretical guarantees for the convergence of the proposed algorithm. Finally, we demonstrate the empirical effectiveness of our method on synthetic and image datasets.

Problem

• Learning latent variable models with energy-based priors
• Solving maximum marginal likelihood estimation problems
• Developing interacting particle algorithms for latent energy models

Innovation

• Interacting particle algorithms for latent energy models
• Continuous-time SDE framework solving MMLE problems
• Discretized algorithm with theoretical convergence guarantees