Algorithms for Sparse LPN and LSPN Against Low-noise

2024-07-27

AI Summary

This paper studies the Low-Noise Sparse Learning Parity with Noise (LSPN) and Sparse LPN problems. Motivated by their foundational role as hardness assumptions in cryptography and the inefficiency of existing algorithms in the low-noise regime, we propose the first unified combinatorial learning framework integrating sparse structure discovery, low-noise exploitation, optimized enumeration, and algebraic reasoning. Our theoretical contributions are: (i) reducing the time complexity of LSPN to $O(eta n/k)^k$, achieving the first improvement over the best-known algorithms for $eta in (sqrt{k/n},, k/n)$; (ii) solving sparse LPN in $e^{O(eta n^{0.7})}$ time using only subquadratic samples $m = n^{1.4}$, which significantly outperforms Gaussian elimination ($e^{eta n}$) when $eta < n^{-0.6}$. All algorithms require only polynomial space.

Technology Category

• Search and Optimization: Learning to Search
• Machine Learning: Probabilistic Circuits and Graphical Models
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Security and Privacy: Applications of cryptography
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

We consider sparse variants of the classical Learning Parities with random Noise (LPN) problem. Our main contribution is a new algorithmic framework that provides learning algorithms against low-noise for both Learning Sparse Parities (LSPN) problem and sparse LPN problem. Different from previous approaches for LSPN and sparse LPN, this framework has a simple structure and runs in polynomial space. Let $n$ be the dimension, $k$ denote the sparsity, and $eta$ be the noise rate. As a fundamental problem in computational learning theory, Learning Sparse Parities with Noise (LSPN) assumes the hidden parity is $k$-sparse. While a simple enumeration algorithm takes ${n choose k}=O(n/k)^k$ time, previously known results stills need ${n choose k/2} = Omega(n/k)^{k/2}$ time for any noise rate $eta$. Our framework provides a LSPN algorithm runs in time $O(eta cdot n/k)^k$ for any noise rate $eta$, which improves the state-of-the-art of LSPN whenever $eta in (sqrt{k/n},k/n)$. The sparse LPN problem is closely related to the classical problem of refuting random $k$-CSP and has been widely used in cryptography as the hardness assumption. Different from the standard LPN, it samples random $k$-sparse vectors. Because the number of $k$-sparse vectors is ${n choose k}<n^k$, sparse LPN has learning algorithms in polynomial time when $m>n^{k/2}$. However, much less is known about learning algorithms for constant $k$ like 3 and $m<n^{k/2}$ samples, except the Gaussian elimination algorithm of time $e^{eta n}$. Our framework provides a learning algorithm in $e^{O(eta cdot n^{frac{delta+1}{2}})}$ time given $delta in (0,1)$ and $m approx n^{1+(1-delta)cdot frac{k-1}{2}}$ samples. This improves previous learning algorithms. For example, in the classical setting of $k=3$ and $m=n^{1.4}$, our algorithm would be faster than $e^{eta n}$ for any $eta<n^{-0.6}$.

Problem

• Develops algorithm for Sparse LPN and LSPN
• Improves efficiency in low-noise conditions
• Enhances computational learning theory methods

Innovation

• New algorithmic framework
• Polynomial space efficiency
• Improved time complexity