Hardness of sampling solutions from the Symmetric Binary Perceptron

2024-07-23

arXiv.org

AI Summary

This work investigates the computational complexity of approximately uniformly sampling solutions from the solution space of the symmetric binary perceptron model. While finding a single solution is efficiently feasible at any constraint density, we prove that no stable algorithm nor bounded-depth Boolean circuit can achieve approximate uniform sampling—establishing the first rigorous separation between “feasible search” and “feasible uniform sampling” within the same NP-hard model. Our approach integrates stability analysis, Boolean circuit complexity theory, and a statistical-physics-inspired constructive reduction. The main contribution is a hardness result demonstrating that uniform sampling remains intractable across the entire constraint-density regime for both stable algorithms and constant-depth Boolean circuits. This provides a critical counterexample to prevailing assumptions in computational phase transition theory, revealing that uniform sampling is fundamentally harder than mere solution existence or search—a qualitative leap in computational difficulty beyond standard NP-hardness.

Technology Category

• Machine Learning: Probabilistic Circuits and Graphical Models
• Constraint Satisfaction and Optimization: Satisfiability
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Economics, Online Markets and Human Computation: Research challenges in human and human-AI computation
• Social Networks and Social Media: Social mining and social search on the Web

Abstract

We show that two related classes of algorithms, stable algorithms and Boolean circuits with bounded depth, cannot produce an approximate sample from the uniform measure over the set of solutions to the symmetric binary perceptron model at any constraint-to-variable density. This result is in contrast to the question of finding emph{a} solution to the same problem, where efficient (and stable) algorithms are known to succeed at sufficiently low density. This result suggests that the solutions found efficiently -- whenever this task is possible -- must be highly atypical, and therefore provides an example of a problem where search is efficiently possible but approximate sampling from the set of solutions is not, at least within these two classes of algorithms.

Problem

• Sampling solutions from symmetric binary perceptron is hard
• Stable algorithms fail to sample solutions uniformly
• Efficient search finds atypical solutions, not samples

Innovation

• Excludes stable algorithms for sampling
• Bans bounded-depth Boolean circuits
• Highlights search-sampling efficiency gap