Almost-Optimal Local-Search Methods for Sparse Tensor PCA

2025-06-11

AI Summary

In sparse tensor principal component analysis (TPCA), conventional local search methods—e.g., Markov Chain Monte Carlo algorithms—suffer from fundamental theoretical limitations, falling significantly short of the statistical accuracy achievable by polynomial-time optimal algorithms. To bridge this gap, we propose a novel class of local search methods: greedy and stochastic greedy algorithms grounded in regularized Bayesian posterior modeling, complemented by a newly introduced Gaussian random thresholding mechanism. This mechanism breaks inter-iteration dependence, enabling the first tight theoretical analysis of stochastic greedy trajectories. We prove that our method attains the state-of-the-art statistical accuracy of polynomial-time optimal algorithms over a broad parameter regime. Empirically, it substantially outperforms classical local search baselines. This work closes the long-standing theoretical gap between computationally efficient local search and polynomial-time optimal estimation, establishing a new paradigm for sparse high-order structure learning that simultaneously achieves computational efficiency and statistical optimality.

Technology Category

• Machine Learning: Matrix & Tensor Methods
• Reasoning under Uncertainty: Stochastic Optimization
• Search and Optimization: Local Search

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Security and Privacy: Large-scale security measurements

Abstract

Local-search methods are widely employed in statistical applications, yet interestingly, their theoretical foundations remain rather underexplored, compared to other classes of estimators such as low-degree polynomials and spectral methods. Of note, among the few existing results recent studies have revealed a significant"local-computational"gap in the context of a well-studied sparse tensor principal component analysis (PCA), where a broad class of local Markov chain methods exhibits a notable underperformance relative to other polynomial-time algorithms. In this work, we propose a series of local-search methods that provably"close"this gap to the best known polynomial-time procedures in multiple regimes of the model, including and going beyond the previously studied regimes in which the broad family of local Markov chain methods underperforms. Our framework includes: (1) standard greedy and randomized greedy algorithms applied to the (regularized) posterior of the model; and (2) novel random-threshold variants, in which the randomized greedy algorithm accepts a proposed transition if and only if the corresponding change in the Hamiltonian exceeds a random Gaussian threshold-rather that if and only if it is positive, as is customary. The introduction of the random thresholds enables a tight mathematical analysis of the randomized greedy algorithm's trajectory by crucially breaking the dependencies between the iterations, and could be of independent interest to the community.

Problem

• Theoretical gaps in local-search methods for sparse tensor PCA
• Underperformance of local Markov chain methods in sparse tensor PCA
• Proposing new local-search methods to close computational gaps

Innovation

• Greedy algorithms on regularized posterior
• Random-threshold variants for transitions
• Breaking dependencies with Gaussian thresholds