ModelVerification.jl: a Comprehensive Toolbox for Formally Verifying Deep Neural Networks

2024-06-30

arXiv.org

AI Summary

Current deep neural network (DNN) formal verification lacks a unified framework capable of jointly verifying diverse network architectures and heterogeneous safety properties—including robustness, fairness, and output bounds. To address this, we propose the first comprehensive toolbox for DNN formal verification: a Julia-based, integrated framework that uniformly supports four verification paradigms—exact verification, approximate solving, abstract interpretation, and interval analysis. We introduce a novel extensible solver interface compatible with SMT solvers (Z3, CVC5), linear programming, symbolic interval arithmetic, and user-defined specifications. Evaluated on MNIST, CIFAR-10, and ACAS Xu benchmarks, our toolbox achieves state-of-the-art verification efficiency. It delivers end-to-end reproducible workflows and provides interpretable, visualization-enhanced diagnostic reports, enabling rigorous, scalable, and customizable DNN verification.

Technology Category

• Machine Learning: Deep Neural Architectures and Foundation Models
• Computer Vision: Adversarial Attacks & Robustness
• Natural Language Processing: Safety and Robustness

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Deep Neural Networks (DNN) are crucial in approximating nonlinear functions across diverse applications, ranging from image classification to control. Verifying specific input-output properties can be a highly challenging task due to the lack of a single, self-contained framework that allows a complete range of verification types. To this end, we present exttt{ModelVerification.jl (MV)}, the first comprehensive, cutting-edge toolbox that contains a suite of state-of-the-art methods for verifying different types of DNNs and safety specifications. This versatile toolbox is designed to empower developers and machine learning practitioners with robust tools for verifying and ensuring the trustworthiness of their DNN models.

Problem

• Lack of unified framework for DNN verification
• Challenges in verifying diverse DNN input-output properties
• Need for versatile toolbox to ensure DNN trustworthiness

Innovation

• Comprehensive toolbox for DNN verification
• Suite of state-of-the-art verification methods
• Ensures DNN trustworthiness via robust tools