Behavioral Audit of Machine Unlearning Has a Privacy Cost

2026-06-12

AI Summary

This study addresses the lack of trustworthy auditing mechanisms in machine unlearning (MU) and demonstrates that behavior-based auditing—under adversarial assumptions where the model owner is dishonest and the auditor is honest but curious—inevitably risks leaking membership information about retained data. For the first time, the authors theoretically prove from an information-theoretic perspective that, in convex models, any effective unlearning audit relying solely on model queries necessarily compromises membership privacy. They further validate through experiments that this fundamental trade-off between auditability and privacy persists in non-convex models as well. Combining rigorous theoretical analysis with empirical evaluation via membership inference attacks, this work exposes the inherent privacy vulnerabilities of behavioral auditing and provides a critical warning for the design of MU mechanisms that must simultaneously ensure verifiability and strong privacy guarantees.

Abstract

The removal of learned data from Machine Learning models through Machine Unlearning (MU) has been widely studied; however, there has yet to be an agreed-upon scheme for auditing MU. Existing work has shown that a dishonest model owner can falsify evidence to avoid executing MU, while curious auditors (and adversaries) can infer the privacy-sensitive properties of the model and its training data even with limited access. Yet auditing of MU under mutual distrust between the model owner and the auditor remains unexplored. We provide an information-theoretic proof for this scenario: for convex ML models, a generic audit scheme that relies solely on querying the model for \textit{behavioral} signals cannot identify insufficiently unlearned models without revealing membership information of the retained set. Therefore, auditing MU under the assumption of a dishonest model owner and an honest-but-curious auditor faces an inherent privacy-audit tradeoff. Our empirical results on convex models strongly supports this result, while further experiments demonstrate that this privacy-audit tension persists in non-convex models. Our results call for a more careful consideration of the privacy-audit tension under a realistic auditor threat model, and serve as a foundation for more scrutiny of designs of privacy-preserving audit schemes for the MU pipeline. We also release our code implementation at https://github.com/LiouTang/Behavioral-Unlearn-Audit.

Problem

• Machine Unlearning
• Behavioral Audit
• Privacy-Audit Tradeoff
• Membership Inference
• Mutual Distrust

Innovation

• Machine Unlearning
• Behavioral Audit
• Privacy-Audit Tradeoff
• Information-Theoretic Bound
• Membership Inference