On the Structure of Stationary Solutions to McKean-Vlasov Equations with Applications to Noisy Transformers

2025-10-22

AI Summary

This study systematically characterizes the structure and phase transition mechanisms of stationary solutions to the McKean–Vlasov equation on the circle. Methodologically, it employs Fourier expansion to recast the stationary problem as an infinite-dimensional quadratic algebraic system, thereby establishing, for the first time, an exact analytical correspondence between stationary solutions and their Fourier coefficients. Building on this, the work rigorously derives explicit closed-form expressions for multimodal stationary states, fully classifies supercritical and subcritical bifurcations, and provides analytic criteria for multistability coexistence and discontinuous (first-order) phase transitions; it further reveals the intrinsic link between nonsmoothness of the free-energy landscape and phase transitions. The approach integrates Fourier analysis, infinite-dimensional bifurcation theory, and variational methods, with special treatment of singular interaction potentials. The results successfully explain noise-strength–driven multimodal emergence and first-order phase transitions in Noisy Transformers, providing the first rigorous phase transition framework for mean-field neural dynamics.

Technology Category

• Machine Learning: Kernel Methods
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Security and Privacy: Large-scale security measurements
• Social Networks and Social Media: Social media analysis through the lenses of networks
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

We study stationary solutions of McKean-Vlasov equations on the circle. Our main contributions stem from observing an exact equivalence between solutions of the stationary McKean-Vlasov equation and an infinite-dimensional quadratic system of equations over Fourier coefficients, which allows explicit characterization of the stationary states in a sequence space rather than a function space. This framework provides a transparent description of local bifurcations, characterizing their periodicity, and resonance structures, while accommodating singular potentials. We derive analytic expressions that characterize the emergence, form and shape (supercritical, critical, subcritical or transcritical) of bifurcations involving possibly multiple Fourier modes and connect them with discontinuous phase transitions. We also characterize, under suitable assumptions, the detailed structure of the stationary bifurcating solutions that are accurate upto an arbitrary number of Fourier modes. At the global level, we establish regularity and concavity properties of the free energy landscape, proving existence, compactness, and coexistence of globally minimizing stationary measures, further identifying discontinuous phase transitions with points of non-differentiability of the minimum free energy map. As an application, we specialize the theory to the Noisy Mean-Field Transformer model, where we show how changing the inverse temperature parameter $β$ affects the geometry of the infinitely many bifurcations from the uniform measure. We also explain how increasing $β$ can lead to a rich class of approximate multi-mode stationary solutions which can be seen as `metastable states'. Further, a sharp transition from continuous to discontinuous (first-order) phase behavior is observed as $β$ increases.

Problem

• Characterizing stationary solutions of McKean-Vlasov equations on circle
• Analyzing bifurcation structures and phase transitions in Fourier space
• Applying theory to understand noisy transformer model dynamics

Innovation

• Equating stationary solutions with Fourier coefficient systems
• Deriving analytic expressions for bifurcation emergence and form
• Establishing free energy landscape regularity and concavity properties