Scalable Neural Network Verification with Branch-and-bound Inferred Cutting Planes

2024-12-31

AI Summary

To address the scalability limitations of cutting-plane methods in formal verification of large neural networks—stemming from their reliance on external mixed-integer programming (MIP) solvers—this paper proposes Branch-and-Bound Inference-based Cut Generation (BICCOS), a dedicated cut-generation mechanism leveraging neuron-level logical relationships within the branch-and-bound search tree. BICCOS integrates constraint strengthening with multi-tree cooperative search to enable neuron-level influence modeling and path-aware cut generation, and is deeply embedded into the α,β-CROWN framework. Evaluated on the VNN-COMP 2024 benchmark, BICCOS substantially increases the number of verifiable instances, marking the first time cutting-plane methods effectively scale to previously intractable large-scale networks. It serves as the core technical component of the competition’s winning verifier.

Technology Category

• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories
• Machine Learning: Calibration & Uncertainty Quantification
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

Recently, cutting-plane methods such as GCP-CROWN have been explored to enhance neural network verifiers and made significant advances. However, GCP-CROWN currently relies on generic cutting planes (cuts) generated from external mixed integer programming (MIP) solvers. Due to the poor scalability of MIP solvers, large neural networks cannot benefit from these cutting planes. In this paper, we exploit the structure of the neural network verification problem to generate efficient and scalable cutting planes specific for this problem setting. We propose a novel approach, Branch-and-bound Inferred Cuts with COnstraint Strengthening (BICCOS), which leverages the logical relationships of neurons within verified subproblems in the branch-and-bound search tree, and we introduce cuts that preclude these relationships in other subproblems. We develop a mechanism that assigns influence scores to neurons in each path to allow the strengthening of these cuts. Furthermore, we design a multi-tree search technique to identify more cuts, effectively narrowing the search space and accelerating the BaB algorithm. Our results demonstrate that BICCOS can generate hundreds of useful cuts during the branch-and-bound process and consistently increase the number of verifiable instances compared to other state-of-the-art neural network verifiers on a wide range of benchmarks, including large networks that previous cutting plane methods could not scale to. BICCOS is part of the $alpha,eta$-CROWN verifier, the VNN-COMP 2024 winner. The code is available at http://github.com/Lemutisme/BICCOS .

Problem

• Neural Network Verification
• Large-scale Models
• Mathematical Tool Limitations

Innovation

• BICCOS
• Neural Network Verification
• Cutting Plane Optimization