🤖 AI Summary
This work addresses the challenges of sampling collapse, negative-phase localization, and parameter drift in Restricted Boltzmann Machine (RBM) training under fixed-temperature finite-time Gibbs sampling. To overcome these issues, the authors propose an endogenous thermodynamic regulation framework that treats temperature as a dynamic state variable coupled with sampling statistics, thereby modeling RBM training as a controlled non-equilibrium dynamical process. By incorporating a thermodynamic self-regulation mechanism, the approach rigorously ensures global parameter boundedness under L2 regularization and local exponential stability of subsystems, effectively preventing inverse-temperature explosion and sampling freeze. Theoretical analysis leverages two-time-scale separation and local Lipschitz conditions. Experiments on MNIST demonstrate significantly improved normalized stability and effective sample size while preserving reconstruction performance, outperforming fixed-temperature baselines.
📝 Abstract
Restricted Boltzmann Machines (RBMs) are typically trained using finite-length Gibbs chains under a fixed sampling temperature. This practice implicitly assumes that the stochastic regime remains valid as the energy landscape evolves during learning. We argue that this assumption can become structurally fragile under finite-time training dynamics. This fragility arises because, in nonconvex energy-based models, fixed-temperature finite-time training can generate admissible trajectories with effective-field amplification and conductance collapse. As a result, the Gibbs sampler may asymptotically freeze, the negative phase may localize, and, without sufficiently strong regularization, parameters may exhibit deterministic linear drift. To address this instability, we introduce an endogenous thermodynamic regulation framework in which temperature evolves as a dynamical state variable coupled to measurable sampling statistics. Under standard local Lipschitz conditions and a two-time-scale separation regime, we establish global parameter boundedness under strictly positive L2 regularization. We further prove local exponential stability of the thermodynamic subsystem and show that the regulated regime mitigates inverse-temperature blow-up and freezing-induced degeneracy within a forward-invariant neighborhood. Experiments on MNIST demonstrate that the proposed self-regulated RBM substantially improves normalization stability and effective sample size relative to fixed-temperature baselines, while preserving reconstruction performance. Overall, the results reinterpret RBM training as a controlled non-equilibrium dynamical process rather than a static equilibrium approximation.