Automated Verification of Soundness of DNN Certifiers

2025-04-06

AI Summary

The opacity of deep neural networks (DNNs) hinders their deployment in safety-critical applications; existing abstract interpretation–based verifiers often suffer from unsoundness—yielding false positives—due to mathematically incomplete logical foundations, and their soundness proofs rely on labor-intensive manual formalization, impeding rapid development of new verifiers. Method: We propose the first automated framework for verifying the soundness of DNN certifiers, built upon symbolic DNN modeling, formal semantics of the ConstraintFlow domain-specific language (DSL), and integration of symbolic execution, SMT solving (via Z3), and abstract interpretation. Contribution/Results: (1) We introduce symbolic DNN modeling, reducing universally quantified soundness properties to SMT-checkable formulas; (2) we formally define ConstraintFlow’s syntax and operational semantics, enabling scalable, architecture-agnostic verification of diverse abstract analyses. Our framework successfully verifies multiple existing and novel certifiers, substantially improving development efficiency and trustworthiness; the implementation is open-sourced.

Technology Category

• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories
• Knowledge Representation and Reasoning: Automated Reasoning and Theorem Proving
• Computer Vision: Visual Reasoning & Symbolic Representations

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics
• Web Mining and Content Analysis: Web data provenance, reliability, and authenticity

Abstract

The uninterpretability of Deep Neural Networks (DNNs) hinders their use in safety-critical applications. Abstract Interpretation-based DNN certifiers provide promising avenues for building trust in DNNs. Unsoundness in the mathematical logic of these certifiers can lead to incorrect results. However, current approaches to ensure their soundness rely on manual, expert-driven proofs that are tedious to develop, limiting the speed of developing new certifiers. Automating the verification process is challenging due to the complexity of verifying certifiers for arbitrary DNN architectures and handling diverse abstract analyses. We introduce ProveSound, a novel verification procedure that automates the soundness verification of DNN certifiers for arbitrary DNN architectures. Our core contribution is the novel concept of a symbolic DNN, using which, ProveSound reduces the soundness property, a universal quantification over arbitrary DNNs, to a tractable symbolic representation, enabling verification with standard SMT solvers. By formalizing the syntax and operational semantics of ConstraintFlow, a DSL for specifying certifiers, ProveSound efficiently verifies both existing and new certifiers, handling arbitrary DNN architectures. Our code is available at https://github.com/uiuc-focal-lab/constraintflow.git

Problem

• Automating soundness verification for DNN certifiers
• Reducing manual effort in certifier development
• Handling arbitrary DNN architectures and abstract analyses

Innovation

• Automates soundness verification for DNN certifiers
• Uses symbolic DNNs for tractable verification
• Formalizes certifier syntax with ConstraintFlow DSL