Natural Identifiers for Privacy and Data Audits in Large Language Models

2026-06-23

AI Summary

This work addresses the challenge of conducting efficient and scalable post-hoc privacy audits on deployed large language models, which existing methods struggle to achieve without either injecting synthetic data during training or requiring private holdout datasets drawn from the same distribution as the training data. To overcome these limitations, the paper introduces Natural Identifiers (NIDs)—structured random strings inherently present in training data, such as hash values or shortened URLs—as intrinsic signals for auditing. Leveraging NIDs, the authors develop the first general-purpose, post-training privacy auditing framework that requires no modification to the training process and no access to private reserved data. The approach supports both differential privacy verification and dataset inference, and its practicality is further enhanced through distribution-consistent synthetic data, enabling effective and scalable privacy evaluation across diverse scenarios.

Abstract

Assessing the privacy of large language models (LLMs) presents significant challenges. In particular, most existing methods for auditing differential privacy require the insertion of specially crafted canary data during training, making them impractical for auditing already-trained models without costly retraining. Additionally, dataset inference, which audits whether a suspect dataset was used to train a model, is infeasible without access to a private non-member held-out dataset. Yet, such held-out datasets are often unavailable or difficult to construct for real-world cases since they have to be from the same distribution (IID) as the suspect data. These limitations severely hinder the ability to conduct scalable, post-hoc audits. To enable such audits, this work introduces natural identifiers (NIDs) as a novel solution to the above-mentioned challenges. NIDs are structured random strings, such as cryptographic hashes and shortened URLs, naturally occurring in common LLM training datasets. Their format enables the generation of unlimited additional random strings from the same distribution, which can act as alternative canaries for audits and as same-distribution held-out data for dataset inference. Our evaluation highlights that indeed, using NIDs, we can facilitate post-hoc differential privacy auditing without any retraining and enable dataset inference for any suspect dataset containing NIDs without the need for a private non-member held-out dataset.

Problem

• privacy auditing
• differential privacy
• dataset inference
• large language models
• post-hoc audit

Innovation

• Natural Identifiers
• Differential Privacy Auditing
• Dataset Inference
• Post-hoc Audit
• Large Language Models