Condition Numbers and Eigenvalue Spectra of Shallow Networks on Spheres

2025-11-04

AI Summary

This work investigates the condition number stability of mass and stiffness matrices induced by shallow ReLU$^k$ neural networks on the unit sphere $mathbb{S}^d$, focusing on antipodally quasi-uniform node sets. It establishes sharp, quantitative estimates of the condition number and its interplay with approximation power and numerical stability. Methodologically, the paper introduces a novel analytical framework unifying harmonic analysis and spherical approximation theory to derive complete asymptotic characterizations of the eigenvalue spectrum: the smallest eigenvalues correspond to low-degree spherical harmonics, while the largest eigenvalues dominate high-degree components. This yields tight, full-spectrum estimates for both extremal eigenvalues. The resulting optimal condition number bound precisely quantifies the fundamental trade-off between expressive capacity and solver stability. These results provide a rigorous theoretical foundation and practical criteria for analyzing and training spherical neural networks.

Technology Category

• Machine Learning: Learning with Manifolds
• Reasoning under Uncertainty: Stochastic Optimization
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding
• Social Networks and Social Media: Social media analysis through the lenses of networks

Abstract

We present an estimation of the condition numbers of the emph{mass} and emph{stiffness} matrices arising from shallow ReLU$^k$ neural networks defined on the unit sphere~$mathbb{S}^d$. In particular, when ${ heta_j^*}_{j=1}^n subset mathbb{S}^d$ is emph{antipodally quasi-uniform}, the condition number is sharp. Indeed, in this case, we obtain sharp asymptotic estimates for the full spectrum of eigenvalues and characterize the structure of the corresponding eigenspaces, showing that the smallest eigenvalues are associated with an eigenbasis of low-degree polynomials while the largest eigenvalues are linked to high-degree polynomials. This spectral analysis establishes a precise correspondence between the approximation power of the network and its numerical stability.

Problem

• Estimating condition numbers of mass and stiffness matrices in shallow neural networks
• Analyzing full eigenvalue spectrum for ReLU networks on spheres
• Establishing relationship between network approximation power and numerical stability

Innovation

• Sharp spectral analysis of shallow ReLU networks
• Estimating condition numbers of mass and stiffness matrices
• Characterizing eigenvalues and eigenspaces on spheres