Strong Low Degree Hardness for the Number Partitioning Problem

2025-05-27

AI Summary

The Number Partitioning Problem (NPP) exhibits a statistical-computational gap under random Gaussian inputs: the optimal solution discrepancy is $2^{-Theta(N)}$, whereas polynomial-time algorithms achieve only $2^{-Theta(log^2 N)}$. Method: We introduce a low-degree polynomial algorithm framework with randomized rounding, characterize solution-space structure via the *conditional overlap gap* (OGP), and integrate solution isolation with noise sensitivity analysis. Contribution/Results: For independent inputs with bounded density, we establish a nearly tight hardness bound: any degree-$D$ algorithm fails to achieve precision better than $2^{-widetilde{O}(D)}$. This implies that $2^{-Theta(log^2 N)}$ is a fundamental barrier for low-complexity algorithms, rigorously supporting the conjecture that “simple exhaustive search is essentially optimal.” Our result is the first near-tight lower bound for NPP in the low-degree framework, bridging statistical optimality and computational tractability.

Technology Category

• Search and Optimization: Non-convex Optimization
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Security and Privacy: Data transparency and provenance

Abstract

In the number partitioning problem (NPP) one aims to partition a given set of $N$ real numbers into two subsets with approximately equal sum. The NPP is a well-studied optimization problem and is famous for possessing a statistical-to-computational gap: when the $N$ numbers to be partitioned are i.i.d. standard gaussian, the optimal discrepancy is $2^{-Theta(N)}$ with high probability, but the best known polynomial-time algorithms only find solutions with a discrepancy of $2^{-Theta(log^2 N)}$. This gap is a common feature in optimization problems over random combinatorial structures, and indicates the need for a study that goes beyond worst-case analysis. We provide evidence of a nearly tight algorithmic barrier for the number partitioning problem. Namely we consider the family of low coordinate degree algorithms (with randomized rounding into the Boolean cube), and show that degree $D$ algorithms fail to solve the NPP to accuracy beyond $2^{-widetilde O(D)}$. According to the low degree heuristic, this suggests that simple brute-force search algorithms are nearly unimprovable, given any allotted runtime between polynomial and exponential in $N$. Our proof combines the isolation of solutions in the landscape with a conditional form of the overlap gap property: given a good solution to an NPP instance, slightly noising the NPP instance typically leaves no good solutions near the original one. In fact our analysis applies whenever the $N$ numbers to be partitioned are independent with uniformly bounded density.

Problem

• Investigates the computational gap in number partitioning problem solutions
• Analyzes limitations of low degree algorithms for accurate NPP solutions
• Explores solution isolation under noise in NPP instances

Innovation

• Low coordinate degree algorithms analysis
• Randomized rounding into Boolean cube
• Conditional overlap gap property proof