Discrepancy Algorithms for the Binary Perceptron

2024-07-19

arXiv.org

AI Summary

This paper studies the asymmetric binary perceptron problem: constructing a sign vector lying in the intersection of random half-spaces with intercept −κ. We are the first to systematically apply classical discrepancy algorithms—including the Lovett–Meka random walk and the Rothvoss/Eldan–Singh geometric partitioning method—to this model, integrating high-dimensional probability theory and statistical physics-inspired phase transition concepts. Our main contributions are threefold: (1) We design efficient constructive algorithms for both κ = 0 and κ → −∞; (2) We precisely characterize the storage capacity in the κ → −∞ regime; (3) We establish an almost-tight lower bound on the overlap gap, thereby providing the first algorithmic evidence that this gap constitutes a fundamental computational bottleneck—bridging the gap between information-theoretic feasibility and constructive tractability.

Technology Category

• Machine Learning: Probabilistic Circuits and Graphical Models
• Constraint Satisfaction and Optimization: Satisfiability
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

The binary perceptron problem asks us to find a sign vector in the intersection of independently chosen random halfspaces with intercept $-kappa$. We analyze the performance of the canonical discrepancy minimization algorithms of Lovett-Meka and Rothvoss/Eldan-Singh for the asymmetric binary perceptron problem. We obtain new algorithmic results in the $kappa = 0$ case and in the large-$|kappa|$ case. In the $kappa o-infty$ case, we additionally characterize the storage capacity and complement our algorithmic results with an almost-matching overlap-gap lower bound.

Problem

• Finding sign vector in random halfspaces intersection
• Analyzing discrepancy algorithms for binary perceptron
• Characterizing storage capacity and algorithmic bounds

Innovation

• Discrepancy minimization for binary perceptron
• Algorithmic analysis for asymmetric cases
• Storage capacity characterization for large intercepts