Phantoms and Disclosures: a Causal Framework for Auditing Synthetic Data

2026-06-15

AI Summary

This work addresses the risk that generative AI models may inadvertently leak private user data from their training sets through synthetic outputs, a concern exacerbated by the difficulty of distinguishing genuine privacy leakage from coincidental matches. To tackle this challenge, the paper introduces the first model-agnostic causal auditing framework that requires only synthetic outputs and a held-out reference dataset. By integrating statistical hypothesis testing with causal inference, the method rigorously differentiates true data leakage from spurious (“phantom”) matches. Applicable to any generative mechanism, the approach operates without shadow models or honeypot data, incurs computational costs orders of magnitude lower than existing techniques, and yields interpretable, tight lower bounds on privacy leakage.

Abstract

The rapid adoption of generative AI and Large Language Models (LLMs) has spurred interest in synthetic data as a privacy-preserving alternative to sensitive real-world datasets. However, generating high-utility synthetic data often carries the risk of memorizing and regurgitating private information from the training corpus. In this work, we present a customizable empirical auditing framework designed to detect and explain such data disclosures. Our framework introduces a mechanism to distinguish between "true disclosures"-where the system directly reproduces a user's information-and "phantom disclosures''-where the system incidentally generates a user's data. By partitioning input data into training and holdout sets and applying rigorous statistical hypothesis testing, we determine if observed disclosures are consistent with strict privacy baselines, such as zero-learning or specific Differential Privacy (DP) bounds. Crucially, this approach requires no model access, no canary insertion, and no reference model training -only the synthetic output and a held-out control set. We demonstrate that this framework effectively functions as a membership inference attack, providing empirical lower bounds on privacy leakage that are tighter than prior data-based auditing methods. Our approach is model-agnostic, applies to any synthetic data generation mechanism, and requires orders of magnitude fewer computational resources than shadow-model or canary-based alternatives.

Problem

• synthetic data
• privacy leakage
• data disclosure
• generative AI
• membership inference

Innovation

• synthetic data auditing
• phantom disclosures
• membership inference
• differential privacy
• model-agnostic privacy evaluation