New Statistical and Computational Results for Learning Junta Distributions

2025-05-09

AI Summary

This paper studies the problem of learning $k$-junta distributions over the Boolean hypercube ${0,1}^n$, i.e., distributions whose probability mass functions depend on at most $k$ variables. It establishes, for the first time, a computational equivalence between junta distribution learning and noisy $k$-parity learning (LPN). Leveraging Fourier analysis, low-order correlation screening, and noise-robust moment estimation, the authors design a statistically optimal and computationally efficient algorithm: it achieves the tight sample complexity $O(2^k log n)$—up to polylogarithmic factors—and matches the runtime of the best prior algorithms. Moreover, they prove that any substantial improvement in sample or time complexity would imply a breakthrough in solving LPN. This work thus settles the theoretical tight bounds for junta distribution learning and introduces a novel paradigm for learning high-dimensional sparse distributions.

Technology Category

• Machine Learning: Learning Theory
• Search and Optimization: Learning to Search
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Security and Privacy: Large-scale security measurements
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

We study the problem of learning junta distributions on ${0, 1}^n$, where a distribution is a $k$-junta if its probability mass function depends on a subset of at most $k$ variables. We make two main contributions: - We show that learning $k$-junta distributions is emph{computationally} equivalent to learning $k$-parity functions with noise (LPN), a landmark problem in computational learning theory. - We design an algorithm for learning junta distributions whose statistical complexity is optimal, up to polylogarithmic factors. Computationally, our algorithm matches the complexity of previous (non-sample-optimal) algorithms. Combined, our two contributions imply that our algorithm cannot be significantly improved, statistically or computationally, barring a breakthrough for LPN.

Problem

• Learning k-junta distributions on binary variables
• Equivalence to learning k-parity functions with noise
• Optimal algorithm for junta distribution learning

Innovation

• Learning k-junta distributions equals LPN problem
• Algorithm achieves optimal statistical complexity
• Computational complexity matches previous non-optimal algorithms