ExpProof : Operationalizing Explanations for Confidential Models with ZKPs

2025-02-06

AI Summary

In adversarial settings, explanation methods are vulnerable to malicious manipulation, undermining model trustworthiness. This paper introduces, for the first time, zero-knowledge proofs (ZKPs) into explainable AI (XAI), proposing a ZKP-amenable variant of LIME that cryptographically verifies explanation authenticity without revealing model parameters. The method is engineered for compatibility with zk-SNARKs and supports both neural networks and random forests. Extensive evaluation across multiple datasets demonstrates sub-millisecond explanation generation latency, manageable verification overhead, and preservation of explanation fidelity. Our core contributions are: (1) the first verifiable explanation framework designed explicitly for adversarial environments; (2) simultaneous guarantees of explanation actionability and model confidentiality; and (3) a novel paradigm for deploying trustworthy AI in high-stakes domains—including finance and healthcare—where integrity and privacy are critical.

Technology Category

• Humans and AI: Explainable AI (XAI) for Human Understanding
• Machine Learning: Adversarial Learning & Robustness
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• Security and Privacy: Data transparency and provenance
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

In principle, explanations are intended as a way to increase trust in machine learning models and are often obligated by regulations. However, many circumstances where these are demanded are adversarial in nature, meaning the involved parties have misaligned interests and are incentivized to manipulate explanations for their purpose. As a result, explainability methods fail to be operational in such settings despite the demand cite{bordt2022post}. In this paper, we take a step towards operationalizing explanations in adversarial scenarios with Zero-Knowledge Proofs (ZKPs), a cryptographic primitive. Specifically we explore ZKP-amenable versions of the popular explainability algorithm LIME and evaluate their performance on Neural Networks and Random Forests.

Problem

• Operationalize explanations in adversarial scenarios
• Use Zero-Knowledge Proofs for trust in machine learning
• Evaluate ZKP-amenable LIME on Neural Networks and Forests

Innovation

• Zero-Knowledge Proofs for explanations
• ZKP-amenable LIME algorithm
• Evaluation on Neural Networks