Privacy-Preserving Model and Preprocessing Verification for Machine Learning

2025-01-14

AI Summary

To address the challenge of trustworthy verification of machine learning preprocessing pipelines in privacy-sensitive settings, this paper proposes the first verification framework integrating local differential privacy (LDP) with model-agnostic interpretability methods (LIME/SHAP). The framework operates without access to raw data or model internals, enabling two verification tasks under strong privacy guarantees: (1) binary classification to determine whether preprocessing is correct, and (2) multi-class identification of specific error types. Its key innovation lies in injecting LDP noise into the feature attribution process of interpretability methods, thereby preserving both privacy and diagnostic interpretability. Experiments on the Diabetes, Adult, and Student Record datasets demonstrate that our ML-based binary classifier significantly outperforms baselines; for multi-class error localization, a threshold-based approach exhibits robust performance. Overall, the framework achieves high privacy preservation, strong robustness against noise and distribution shifts, and practical applicability.

Technology Category

• Machine Learning: Privacy
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models
• Humans and AI: Human-in-the-loop Machine Learning

Application Category

• Security and Privacy: Data transparency and provenance
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

This paper presents a framework for privacy-preserving verification of machine learning models, focusing on models trained on sensitive data. Integrating Local Differential Privacy (LDP) with model explanations from LIME and SHAP, our framework enables robust verification without compromising individual privacy. It addresses two key tasks: binary classification, to verify if a target model was trained correctly by applying the appropriate preprocessing steps, and multi-class classification, to identify specific preprocessing errors. Evaluations on three real-world datasets-Diabetes, Adult, and Student Record-demonstrate that while the ML-based approach is particularly effective in binary tasks, the threshold-based method performs comparably in multi-class tasks. Results indicate that although verification accuracy varies across datasets and noise levels, the framework provides effective detection of preprocessing errors, strong privacy guarantees, and practical applicability for safeguarding sensitive data.

Problem

• Privacy-Preserving
• Data-Preprocessing
• Error-Detection

Innovation

• Local Differential Privacy
• Model Auditing
• Privacy-Preserving Explanations