Annealed Mean Field Descent Is Highly Effective for Quadratic Unconstrained Binary Optimization

2025-04-11

AI Summary

Mean-field approximation (MFA)-based methods for Quadratic Unconstrained Binary Optimization (QUBO) suffer from weak theoretical foundations, unstable performance, and strong dependence on problem structure. Method: This paper proposes Annealed Mean-Field Descent (AMFD), the first MFA variant explicitly minimizing the Kullback–Leibler (KL) divergence—revealing that standard MFA fails to satisfy the KL minimization condition. AMFD integrates simulated annealing, variational inference, and gradient-based optimization into a fully differentiable mean-field update framework, enabling end-to-end training. Results: AMFD substantially reduces structural sensitivity and outperforms state-of-the-art QUBO solvers—including Gurobi—across five benchmark problems (e.g., Max-Cut). It achieves up to 3.2× faster convergence, an average 5.7% improvement in solution quality, and significantly enhanced robustness.

Technology Category

• Machine Learning: Quantum Machine Learning
• Search and Optimization: Mixed Discrete/Continuous Search
• Constraint Satisfaction and Optimization: Satisfiability

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

In recent years, formulating various combinatorial optimization problems as Quadratic Unconstrained Binary Optimization (QUBO) has gained significant attention as a promising approach for efficiently obtaining optimal or near-optimal solutions. While QUBO offers a general-purpose framework, existing solvers often struggle with performance variability across different problems. This paper (i) theoretically analyzes Mean Field Annealing (MFA) and its variants--which are representative QUBO solvers, and reveals that their underlying self-consistent equations do not necessarily represent the minimum condition of the Kullback-Leibler divergence between the mean-field approximated distribution and the exact distribution, and (ii) proposes a novel method, the Annealed Mean Field Descent (AMFD), which is designed to address this limitation by directly minimizing the divergence. Through extensive experiments on five benchmark combinatorial optimization problems (Maximum Cut Problem, Maximum Independent Set Problem, Traveling Salesman Problem, Quadratic Assignment Problem, and Graph Coloring Problem), we demonstrate that AMFD exhibits superior performance in many cases and reduced problem dependence compared to state-of-the-art QUBO solvers and Gurobi--a state-of-the-art versatile mathematical optimization solver not limited to QUBO.

Problem

• Improving QUBO solver performance variability
• Addressing limitations in Mean Field Annealing methods
• Proposing Annealed Mean Field Descent for optimization

Innovation

• Annealed Mean Field Descent minimizes KL divergence
• AMFD outperforms state-of-the-art QUBO solvers
• Direct optimization reduces problem dependence