Adaptive Branch-and-Bound Tree Exploration for Neural Network Verification

2025-05-02

AI Summary

Traditional branch-and-bound (BaB) methods in neural network formal verification suffer from inefficient, blind exploration of subproblems. Method: This paper proposes an adaptive tree search mechanism that dynamically prioritizes subproblems based on their estimated importance—formally defined as the probability of containing a counterexample—integrating Monte Carlo Tree Search (MCTS) principles to guide intelligent, progressive BaB exploration while preserving both early-termination capability and soundness. Contribution/Results: Evaluated on 552 standard verification benchmarks, our approach achieves up to 15.2× and 24.7× speedup over state-of-the-art verifiers on MNIST and CIFAR-10, respectively, significantly advancing verification efficiency without compromising completeness or correctness guarantees.

Technology Category

• Search and Optimization: Adversarial Search
• Machine Learning: Calibration & Uncertainty Quantification
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Formal verification is a rigorous approach that can provably ensure the quality of neural networks, and to date, Branch and Bound (BaB) is the state-of-the-art that performs verification by splitting the problem as needed and applying off-the-shelf verifiers to sub-problems for improved performance. However, existing BaB may not be efficient, due to its naive way of exploring the space of sub-problems that ignores the emph{importance} of different sub-problems. To bridge this gap, we first introduce a notion of ``importance'' that reflects how likely a counterexample can be found with a sub-problem, and then we devise a novel verification approach, called ABONN, that explores the sub-problem space of BaB adaptively, in a Monte-Carlo tree search (MCTS) style. The exploration is guided by the ``importance'' of different sub-problems, so it favors the sub-problems that are more likely to find counterexamples. As soon as it finds a counterexample, it can immediately terminate; even though it cannot find, after visiting all the sub-problems, it can still manage to verify the problem. We evaluate ABONN with 552 verification problems from commonly-used datasets and neural network models, and compare it with the state-of-the-art verifiers as baseline approaches. Experimental evaluation shows that ABONN demonstrates speedups of up to $15.2 imes$ on MNIST and $24.7 imes$ on CIFAR-10. We further study the influences of hyperparameters to the performance of ABONN, and the effectiveness of our adaptive tree exploration.

Problem

• Improving efficiency of Branch-and-Bound for neural network verification
• Adaptively exploring sub-problems based on importance to find counterexamples
• Enhancing verification speed via Monte-Carlo tree search-guided exploration

Innovation

• Adaptive Branch-and-Bound guided by sub-problem importance
• Monte-Carlo tree search for efficient sub-problem exploration
• Early termination upon counterexample discovery