Dataset Usage Inference without Shadow Models or Held-out Data

2026-06-24

AI Summary

This work addresses the limitations of existing membership inference approaches, which rely on shadow models and held-out ground-truth data—assumptions that rarely hold in real-world scenarios. To overcome this, the authors propose a novel method that requires neither shadow models nor reserved real data. Their approach generates synthetic non-member samples, extracts diverse membership signals, and formulates the problem as mixture proportion estimation to accurately quantify the extent to which training data is utilized by large-scale image generative models. This is the first technique to eliminate dependence on both shadow models and authentic held-out data, offering practical utility in real-world disputes over data ownership and providing data owners with an effective tool for verifying model usage of their data.

Abstract

How much of my data was used to train a machine learning model? Dataset Usage Inference (DUI) aims to answer this by estimating what fraction of a dataset contributed to a model's training. However, existing DUI methods rely on assumptions that rarely hold in practice: they require training expensive shadow models to imitate the target model, and they assume access to both known training samples and an in-distribution held-out set confirmed to be absent from training. These conditions make current approaches impractical for modern large models and real data ownership disputes. We introduce a practical DUI framework that removes these constraints. Our method requires neither shadow models nor real held-out data. Instead, it generates synthetic non-member samples, extracts diverse membership signals, and casts DUI as a mixture proportion estimation problem to estimate what share of the candidate dataset was used during training. Experiments on large image generative models show that our method reliably quantifies dataset usage, providing a practical tool for data owners to determine how much of their data was used to train a model.

Problem

• Dataset Usage Inference
• Data Ownership
• Membership Inference
• Mixture Proportion Estimation
• Model Transparency

Innovation

• Dataset Usage Inference
• Shadow Model-Free
• Synthetic Non-Member Generation
• Mixture Proportion Estimation
• Membership Inference