Geometric Dyson Brownian Motions and the Free Log-Normal Limit for a Non-Square Product of Random Matrices

2026-06-29

AI Summary

This work investigates the squared singular value spectrum of products of non-square random matrices arising in the initialization of deep linear neural networks, which corresponds to the limiting eigenvalue distribution of the associated feature covariance matrix. By employing a double asymptotic analysis—first taking depth and width to grow proportionally at fixed sample size, yielding a geometric Dyson Brownian motion, and then passing to the mean-field limit to obtain a T-transform satisfying Burgers’ equation—the study establishes, for the first time, a connection between the spectral dynamics of non-square matrix products and geometric Dyson Brownian motion. This framework leads to the free log-normal law as the limiting spectral distribution, for which an explicit expression, support formula, and a short-time Marchenko–Pastur approximation are derived, alongside an efficient numerical iteration scheme. Theoretical predictions show excellent agreement with finite-dimensional simulations and are successfully applied to risk analysis in random feature regression.

Abstract

We study the squared singular value spectrum of a product of non-square random matrices, a setting that also corresponds to the feature covariance eigenvalues of a deep linear neural network at initialization. We first take a proportional depth-width $d,n$ limit with the number of data points $m$ held fixed, and show that the resulting covariance eigenvalue process satisfies a geometric version of Dyson Brownian motion. We then take a second, sequential mean-field limit corresponding to the scaling $dm/n\to\barτ$, and show that the limiting $T$-transform of the spectrum solves a Burgers equation. In the identity-start case this equation yields the free log-normal law, and the general limit is obtained by free multiplicative convolution with the free log-normal. We further obtain the free log-normal support formula, a fixed-point iteration for numerical evaluation, and a formal small-time Marchenko--Pastur approximation. We also use the limiting spectral law to predict a toy random-feature regression risk, finding close agreement with a finite-dimensional simulation.

Problem

• random matrices
• singular value spectrum
• Dyson Brownian motion
• free log-normal distribution
• neural network initialization

Innovation

• Geometric Dyson Brownian Motion
• Free Log-Normal Law
• Random Matrix Product
• Burgers Equation
• Free Multiplicative Convolution