The Cost of Relaxation: Evaluating the Error in Convex Neural Network Verification

2026-04-20

AI Summary

This work addresses the unquantified error introduced by convex relaxation methods in neural network verification, which sacrifice completeness for tractability. For the first time, it establishes analytical upper and lower bounds on this error, revealing that it grows exponentially with network depth and linearly with the input perturbation radius. The study further uncovers a step-like behavior in misclassification probability. By constructing a lattice structure in the relaxed space and integrating constraint programming, ℓ∞-norm analysis, and lattice theory, the authors systematically characterize the worst-case deviation between the fully relaxed and original network outputs. Experiments on MNIST, Fashion-MNIST, and randomly generated networks validate the theoretical bounds, providing a quantitative foundation for assessing the reliability of verification methods based on convex relaxation.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Computer Vision: Adversarial Attacks & Robustness
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Social Networks and Social Media: Social media analysis through the lenses of networks
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Web Mining and Content Analysis: Web data provenance, reliability, and authenticity

Abstract

Many neural network (NN) verification systems represent the network's input-output relation as a constraint program. Sound and complete, representations involve integer constraints, for simulating the activations. Recent works convexly relax the integer constraints, improving performance, at the cost of soundness. Convex relaxations consider outputs that are unreachable by the original network. We study the worst case divergence between the original network and its convex relaxations; both qualitatively and quantitatively. The relaxations' space forms a lattice, where the top element corresponds to a full relaxation, with every neuron linearized. The bottom element corresponds to the original network. We provide analytical upper and lower bounds for the $\ell_\infty$-distance between the fully relaxed and original outputs. This distance grows exponentially, w.r.t. the network's depth, and linearly w.r.t. the input's radius. The misclassification probability exhibits a step-like behavior, w.r.t. input radius. Our results are supported by experiments on MNIST, Fashion MNIST and random networks.

Problem

• neural network verification
• convex relaxation
• verification error
• output divergence
• soundness

Innovation

• convex relaxation
• neural network verification
• relaxation error
• ℓ∞-distance bound
• soundness-completeness tradeoff