On Learning Parallel Pancakes with Mostly Uniform Weights

2025-04-21

AI Summary

This work studies the computational complexity of learning $k$-Gaussian mixture models ($k$-GMMs) in $mathbb{R}^d$, focusing on the case where components share a common covariance matrix and weights are not exponentially small—particularly when approximately uniform. First, it establishes a tight statistical query (SQ) lower bound, proving that learning uniform-weight GMMs requires $d^{Omega(log(1/w_{min}))}$ time—implying that quasi-polynomial time is inherent and unavoidable. Second, it introduces a novel SQ-based testing algorithm applicable to a generalized setting where most weights are near-uniform and only a few may be arbitrary. Under this condition, the algorithm achieves testing in $d^{O(log k)}$ quasi-polynomial time, substantially broadening the class of GMMs efficiently learnable in the SQ model. The core contribution is a precise, optimal characterization of the time complexity for GMM learning in high dimensions, breaking the restrictive assumption of strict weight uniformity required by prior work.

Technology Category

• Machine Learning: Quantum Machine Learning
• Search and Optimization: Learning to Search
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Querying, indexing, and retrieval in Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Security and Privacy: Large-scale security measurements

Abstract

We study the complexity of learning $k$-mixtures of Gaussians ($k$-GMMs) on $mathbb{R}^d$. This task is known to have complexity $d^{Omega(k)}$ in full generality. To circumvent this exponential lower bound on the number of components, research has focused on learning families of GMMs satisfying additional structural properties. A natural assumption posits that the component weights are not exponentially small and that the components have the same unknown covariance. Recent work gave a $d^{O(log(1/w_{min}))}$-time algorithm for this class of GMMs, where $w_{min}$ is the minimum weight. Our first main result is a Statistical Query (SQ) lower bound showing that this quasi-polynomial upper bound is essentially best possible, even for the special case of uniform weights. Specifically, we show that it is SQ-hard to distinguish between such a mixture and the standard Gaussian. We further explore how the distribution of weights affects the complexity of this task. Our second main result is a quasi-polynomial upper bound for the aforementioned testing task when most of the weights are uniform while a small fraction of the weights are potentially arbitrary.

Problem

• Study complexity of learning k-GMMs with uniform weights
• SQ lower bound for distinguishing mixtures from Gaussian
• Quasi-polynomial upper bound with mostly uniform weights

Innovation

• SQ lower bound for uniform weight GMMs
• Quasi-polynomial algorithm for mostly uniform weights
• Study of weight distribution impact on complexity