🤖 AI Summary
The black-box nature of neural network (NN) controllers hinders their formal verification in safety-critical systems. To address this, we propose a precise, fully automated transformation method that equivalently converts discrete-output NN controllers—comprising ReLU activations followed by argmax—to soft decision trees (SDTs). This is the first approach to achieve strict functional equivalence for NNs containing argmax layers. Our method integrates symbolic propagation with branch equivalence analysis to automatically prune redundant execution paths. Crucially, the transformation preserves input–output behavior while drastically improving verifiability: on MountainCar-v0 and CartPole-v1, formal verification throughput increases by 21× and 2×, respectively. Moreover, the resulting SDT controllers enhance both verification efficiency and interpretability without compromising correctness.
📝 Abstract
Over the past decade, neural network (NN)-based controllers have demonstrated remarkable efficacy in a variety of decision-making tasks. However, their black-box nature and the risk of unexpected behaviors and surprising results pose a challenge to their deployment in real-world systems with strong guarantees of correctness and safety. We address these limitations by investigating the transformation of NN-based controllers into equivalent soft decision tree (SDT)-based controllers and its impact on verifiability. Differently from previous approaches, we focus on discrete-output NN controllers including rectified linear unit (ReLU) activation functions as well as argmax operations. We then devise an exact but cost-effective transformation algorithm, in that it can automatically prune redundant branches. We evaluate our approach using two benchmarks from the OpenAI Gym environment. Our results indicate that the SDT transformation can benefit formal verification, showing runtime improvements of up to 21x and 2x for MountainCar-v0 and CartPole-v1, respectively.
