Tighter Truncated Rectangular Prism Approximation for RNN Robustness Verification

2025-11-12

AI Summary

Existing robustness verification methods for RNNs suffer from over-approximation when linearly relaxing nonlinear activation functions—particularly the Hadamard product—leading to overly conservative verification results. Method: This paper proposes the Truncated Rectangular Prism (TRP) approximation, which tightly encloses the 3D nonlinear surface using two minimal linear bounding planes and jointly optimizes both prism volume and surface area via a refinement-driven strategy to enhance relaxation tightness. The resulting nonlinear constraints are reformulated as efficiently solvable linear programs, unifying theoretical verifiability with computational tractability. Contribution/Results: We implement DeepPrism, a prototype system based on TRP, and evaluate it on image classification, speech recognition, and sentiment analysis tasks. Compared to state-of-the-art approaches, DeepPrism achieves an average 12.7% improvement in verified accuracy and reduces verification time by 31.4%.

Technology Category

• Natural Language Processing: Safety and Robustness
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Security and Privacy: Large-scale security measurements

Abstract

Robustness verification is a promising technique for rigorously proving Recurrent Neural Networks (RNNs) robustly. A key challenge is to over-approximate the nonlinear activation functions with linear constraints, which can transform the verification problem into an efficiently solvable linear programming problem. Existing methods over-approximate the nonlinear parts with linear bounding planes individually, which may cause significant over-estimation and lead to lower verification accuracy. In this paper, in order to tightly enclose the three-dimensional nonlinear surface generated by the Hadamard product, we propose a novel truncated rectangular prism formed by two linear relaxation planes and a refinement-driven method to minimize both its volume and surface area for tighter over-approximation. Based on this approximation, we implement a prototype DeepPrism for RNN robustness verification. The experimental results demonstrate that emph{DeepPrism} has significant improvement compared with the state-of-the-art approaches in various tasks of image classification, speech recognition and sentiment analysis.

Problem

• Over-approximating nonlinear RNN activations with linear constraints
• Reducing significant over-estimation in existing verification methods
• Tightly enclosing 3D nonlinear surfaces for improved verification accuracy

Innovation

• Truncated rectangular prism for 3D nonlinear surface approximation
• Two linear relaxation planes with refinement-driven volume minimization
• Tighter over-approximation for improved RNN verification accuracy