Deep generative modelling of canonical ensemble with differentiable thermal properties

2024-04-29

arXiv.org

AI Summary

Efficient and accurate computation of thermodynamic quantities in many-body systems has long been hindered by slow Markov chain Monte Carlo (MCMC) convergence and limitations of existing deep learning approaches—including poor temperature generalization and significant sampling bias. This work introduces the first paradigm that embeds temperature as a differentiable parameter into a variational generative model, directly modeling the temperature-dependent Boltzmann distribution without requiring training data. The model is trained end-to-end to minimize the free energy functional. Consequently, thermodynamic observables—such as specific heat and magnetic susceptibility—emerge as analytically differentiable functions of temperature, enabling precise characterization of critical behavior near phase transitions. On the Ising and XY models, our method achieves sampling quality comparable to MCMC but with substantially improved speed. Crucially, second derivatives of the free energy match exact solutions, marking the first demonstration of unbiased, multi-temperature, and physically self-consistent thermodynamic inference within a generative modeling framework.

Technology Category

• Machine Learning: Deep Generative Models & Autoencoders
• Natural Language Processing: Generation
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications

Abstract

We propose a variational modelling method with differentiable temperature for canonical ensembles. Using a deep generative model, the free energy is estimated and minimized simultaneously in a continuous temperature range. At optimal, this generative model is a Boltzmann distribution with temperature dependence. The training process requires no dataset, and works with arbitrary explicit density generative models. We applied our method to study the phase transitions (PT) in the Ising and XY models, and showed that the direct-sampling simulation of our model is as accurate as the Markov Chain Monte Carlo (MCMC) simulation, but more efficient. Moreover, our method can give thermodynamic quantities as differentiable functions of temperature akin to an analytical solution. The free energy aligns closely with the exact one to the second-order derivative, so this inclusion of temperature dependence enables the otherwise biased variational model to capture the subtle thermal effects at the PTs. These findings shed light on the direct simulation of physical systems using deep generative models

Problem

• Computes thermodynamic quantities of many-body systems efficiently.
• Enables direct sampling across temperatures without bias.
• Models phase transitions accurately with differentiable thermal properties.

Innovation

• Variational method for canonical ensembles with differentiable temperature
• General framework compatible with any tractable density generative model
• Direct-sampling simulations accurate as Markov chain Monte Carlo but more efficient