Which exceptional low-dimensional projections of a Gaussian point cloud can be found in polynomial time?

2024-06-05

arXiv.org

AI Summary

This work investigates which $m$-dimensional projections of high-dimensional Gaussian point clouds are efficiently computable by polynomial-time algorithms under the proportional asymptotic regime ($n/d o alpha$). Addressing the challenge that conventional methods fail to characterize the set of algorithmically attainable non-Gaussian projections, we establish, for the first time, a rigorous equivalence between the “computable projection set” and solutions to stochastic optimal control problems governed by the generalized Parisi variational principle—thereby providing a mathematically sound foundation for statistical-physics-inspired formulations. Leveraging tools from random matrix theory, large deviations analysis, and convergence analysis of iterative algorithms, we fully characterize necessary and sufficient conditions for algorithmic realizability of projections. As corollaries, we derive exact computational thresholds for canonical random optimization problems—including the generalized spherical perceptron—uncovering fundamental connections between computational complexity and underlying statistical structure.

Technology Category

• Reasoning under Uncertainty: Stochastic Optimization
• Machine Learning: Probabilistic Circuits and Graphical Models
• Search and Optimization: Non-convex Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Security and Privacy: Large-scale security measurements
• Social Networks and Social Media: Computational social science

Abstract

Given $d$-dimensional standard Gaussian vectors $oldsymbol{x}_1,dots, oldsymbol{x}_n$, we consider the set of all empirical distributions of its $m$-dimensional projections, for $m$ a fixed constant. Diaconis and Freedman (1984) proved that, if $n/d o infty$, all such distributions converge to the standard Gaussian distribution. In contrast, we study the proportional asymptotics, whereby $n,d o infty$ with $n/d o alpha in (0, infty)$. In this case, the projection of the data points along a typical random subspace is again Gaussian, but the set $mathscr{F}_{m,alpha}$ of all probability distributions that are asymptotically feasible as $m$-dimensional projections contains non-Gaussian distributions corresponding to exceptional subspaces. Non-rigorous methods from statistical physics yield an indirect characterization of $mathscr{F}_{m,alpha}$ in terms of a generalized Parisi formula. Motivated by the goal of putting this formula on a rigorous basis, and to understand whether these projections can be found efficiently, we study the subset $mathscr{F}^{ m alg}_{m,alpha}subseteq mathscr{F}_{m,alpha}$ of distributions that can be realized by a class of iterative algorithms. We prove that this set is characterized by a certain stochastic optimal control problem, and obtain a dual characterization of this problem in terms of a variational principle that extends Parisi's formula. As a byproduct, we obtain computationally achievable values for a class of random optimization problems including `generalized spherical perceptron' models.

Problem

• Identify exceptional low-dimensional Gaussian projections efficiently
• Characterize feasible non-Gaussian distributions in proportional asymptotics
• Develop rigorous computational methods for random optimization problems

Innovation

• Studies low-dimensional Gaussian projections asymptotics
• Uses generalized Parisi formula characterization
• Develops iterative algorithms for feasible distributions