Rethinking the Capacity of Graph Neural Networks for Branching Strategy

2024-02-11

arXiv.org

AI Summary

This work investigates the expressive power of graph neural networks (GNNs) in modeling strong branching (SB) scores for mixed-integer linear programming (MILP). We identify a fundamental limitation of standard message-passing GNNs (MP-GNNs): they can only universally approximate SB scores over the restricted subclass of “MP-solvable” MILPs, failing to cover the full MILP space. To address this, we formally define the MP-solvable MILP class and establish a universal approximation theorem for MP-GNNs—along with its precise failure boundary. We then propose the second-order folklore GNN (2-FGNN), and theoretically prove its universal approximation capability for SB scores across the entire MILP space. Numerical experiments on small-scale instances empirically validate the theoretical findings. This work provides the first rigorous, hierarchical characterization of GNN expressivity for MILP branching policies—and identifies a concrete architectural pathway to overcome inherent limitations of MP-GNNs in combinatorial optimization.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Graph neural networks (GNNs) have been widely used to predict properties and heuristics of mixed-integer linear programs (MILPs) and hence accelerate MILP solvers. This paper investigates the capacity of GNNs to represent strong branching (SB), the most effective yet computationally expensive heuristic employed in the branch-and-bound algorithm. In the literature, message-passing GNN (MP-GNN), as the simplest GNN structure, is frequently used as a fast approximation of SB and we find that not all MILPs's SB can be represented with MP-GNN. We precisely define a class of"MP-tractable"MILPs for which MP-GNNs can accurately approximate SB scores. Particularly, we establish a universal approximation theorem: for any data distribution over the MP-tractable class, there always exists an MP-GNN that can approximate the SB score with arbitrarily high accuracy and arbitrarily high probability, which lays a theoretical foundation of the existing works on imitating SB with MP-GNN. For MILPs without the MP-tractability, unfortunately, a similar result is impossible, which can be illustrated by two MILP instances with different SB scores that cannot be distinguished by any MP-GNN, regardless of the number of parameters. Recognizing this, we explore another GNN structure called the second-order folklore GNN (2-FGNN) that overcomes this limitation, and the aforementioned universal approximation theorem can be extended to the entire MILP space using 2-FGNN, regardless of the MP-tractability. A small-scale numerical experiment is conducted to directly validate our theoretical findings.

Problem

• Graph Neural Networks
• Mixed Integer Linear Programming
• Strong Branching

Innovation

• Graph Neural Networks
• Mixed Integer Linear Programming
• Enhanced GNN for SB Strategies