Phase Transitions in the Fluctuations of Functionals of Random Neural Networks

2026-04-21

AI Summary

This work investigates the asymptotic behavior and phase transition mechanisms of output functionals in infinitely wide random neural networks as depth increases. By linking the fixed-point properties of the covariance iteration operator to functional limit behavior, the study uncovers three distinct phase transition regimes governed by the stability of these fixed points. The analysis integrates Hermite expansions, diagrammatic formulas, and Stein–Malliavin theory, introducing a novel perspective through fixed-point analysis of covariance operators. The main contributions are central and non-central limit theorems that rigorously characterize three asymptotic regimes: convergence of the output functional—depending on the underlying covariance structure—to a Gaussian random field, a Gaussian distribution, or a Q-th order Wiener chaos.

Technology Category

• Machine Learning: Kernel Methods
• Reasoning under Uncertainty: Stochastic Optimization
• Search and Optimization: Non-convex Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data
• Social Networks and Social Media: Computational social science

Abstract

We establish central and non-central limit theorems for sequences of functionals of the Gaussian output of an infinitely-wide random neural network on the d-dimensional sphere . We show that the asymptotic behaviour of these functionals as the depth of the network increases depends crucially on the fixed points of the covariance function, resulting in three distinct limiting regimes: convergence to the same functional of a limiting Gaussian field, convergence to a Gaussian distribution, convergence to a distribution in the Qth Wiener chaos. Our proofs exploit tools that are now classical (Hermite expansions, Diagram Formula, Stein-Malliavin techniques), but also ideas which have never been used in similar contexts: in particular, the asymptotic behaviour is determined by the fixed-point structure of the iterative operator associated with the covariance, whose nature and stability governs the different limiting regimes.

Problem

• Phase Transitions
• Random Neural Networks
• Limit Theorems
• Gaussian Fields
• Wiener Chaos

Innovation

• phase transitions
• random neural networks
• Wiener chaos
• covariance fixed points
• limit theorems