Learning as Observable Matrix Dynamics: Diffusive Relaxations versus Phase Transitions

2026-06-28

AI Summary

Traditional scalar loss functions struggle to capture the geometric evolution of high-dimensional representations during neural network training. This work proposes a framework extending BBS theory through an environment–latent decomposition, enabling fine-grained diagnosis of representational geometry via dynamic distance matrices, random matrix theory, and spectral analysis, complemented by trajectory-level observables and three-dimensional MDS embeddings. Across seven experimental settings, the approach successfully distinguishes diffusive relaxation—characterized by the absence of stable spectral bands—from endogenous or exogenously driven reorganization, which exhibits stable spectral fingerprints. These findings reveal distinct latent geometric structures underlying different learning mechanisms, offering insights beyond those provided by scalar measures such as intrinsic dimensionality alone.

Abstract

Observable Matrix Dynamics (OMD) is a diagnostic framework that probes the dynamics of high-dimensional internal representations of inputs by a neural network via a fixed-size $N \times N$ distance matrix $M(t)$ on a held set of $N$ inputs. OMD uses methods of random matrix theory and particle dynamics to explore spectral reorganisations that are missed by scalar loss functions, but are informative of the training process. We read $M(t)$ against a perturbative ambient-versus-latent decomposition extending the Bogomolny--Bohigas--Schmit (BBS) theory of random distance matrices, with per-snapshot diagnostics for the top-of-spectrum band structure and ambient noise, trajectory-level observables linking snapshots, and a 3D MDS embedding (bottom-three eigenvectors) rendering training as a moving particle cloud. Across seven experiments, diffusive regimes lack stable top-of-spectrum band structure, while sharp endogenous or externally driven reorganisations produce stable fingerprints: consistent with smooth or product latent geometries in BBS-adjacent cases, and with finite-cluster or Fourier-soliton structures otherwise. OMD thus reads the geometric regime of a representation rather than reporting a single intrinsic dimension.

Problem

• Observable Matrix Dynamics
• neural network representations
• spectral reorganization
• geometric regime
• random matrix theory

Innovation

• Observable Matrix Dynamics
• Random Matrix Theory
• Spectral Reorganization
• Geometric Regime Detection
• Neural Representation Dynamics