Neural Network Verification using Partial Multi-Neuron Relaxation

2026-05-28

AI Summary

Existing neural network verification methods struggle to balance the scalability of single-neuron relaxations with the tightness of full multi-neuron relaxations, limiting both verification efficiency and success rates. This work proposes a partial multi-neuron relaxation approach that employs heuristic strategies to select critical subsets of neurons, constructs multi-neuron linear bounds, and integrates branch-and-bound with hyperplane optimization. By doing so, it achieves significantly improved scalability while preserving high bound tightness. Experimental evaluation based on the Marabou verifier demonstrates that the proposed method outperforms existing bound-tightening techniques across multiple benchmarks, effectively enhancing both the success rate and computational efficiency of formal verification.

Abstract

The increasing integration of deep neural networks in critical systems has spawned a theoretical and practical interest in formally guaranteeing safety properties about their behavior. To achieve this, contemporary verification algorithms rely on computing linear relaxations for a network's non-linear activation functions. Existing approaches for linear relaxations typically fall into one of two categories: single-neuron relaxation, in which each activation neuron is bounded in terms of its sources; and multi-neuron relaxation, in which linear bounds involving multiple activation neurons and their sources are calculated. However, existing methods might fail to balance tightness and scalability, as single-neuron bounds might not derive sufficiently tight bounds necessary for verification to complete, whereas generating multi-neuron relaxation for all activation neurons is computationally expensive. In this paper, we present a middle-ground approach featuring partial multi-neuron relaxation, in which we generate multi-neuron bounds for only a small, heuristically selected subset of neurons. To achieve this, we build upon existing branching heuristics for selecting neurons and for optimizing bounding hyper-planes for multi-neuron bounds. We integrated our proposed method within the Marabou verifier, and obtained favorable results in comparison to existing bound tightening methods. Our experiments showcase the potential of our technique for neural network verification.

Problem

• neural network verification
• linear relaxation
• single-neuron relaxation
• multi-neuron relaxation
• scalability

Innovation

• partial multi-neuron relaxation
• neural network verification
• linear relaxation
• branching heuristics
• bound tightening