Hey, That's My Data! Label-Only Dataset Inference in Large Language Models

2025-06-06

AI Summary

This paper addresses the challenge of protecting copyright in large language model (LLM) training data by proposing a private-data usage detection method that requires no access to internal model logits—crucial when models withhold log-probability outputs. We introduce CatShift, the first paradigm leveraging catastrophic forgetting: it statistically detects unauthorized use of proprietary datasets by measuring label-level output distribution shifts before and after fine-tuning. CatShift integrates label uniqueness modeling, cross-dataset output comparison, and rigorous hypothesis testing—entirely avoiding reliance on logits or gradients. Evaluated on both open-weight and API-accessible LLMs, CatShift achieves significantly higher accuracy than existing baselines under logit-inaccessible conditions, demonstrating strong robustness and practical deployability. It provides the first production-ready black-box solution for data provenance verification in LLMs.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Computer Vision: Large Vision Models

Application Category

• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Large Language Models (LLMs) have revolutionized Natural Language Processing by excelling at interpreting, reasoning about, and generating human language. However, their reliance on large-scale, often proprietary datasets poses a critical challenge: unauthorized usage of such data can lead to copyright infringement and significant financial harm. Existing dataset-inference methods typically depend on log probabilities to detect suspicious training material, yet many leading LLMs have begun withholding or obfuscating these signals. This reality underscores the pressing need for label-only approaches capable of identifying dataset membership without relying on internal model logits. We address this gap by introducing CatShift, a label-only dataset-inference framework that capitalizes on catastrophic forgetting: the tendency of an LLM to overwrite previously learned knowledge when exposed to new data. If a suspicious dataset was previously seen by the model, fine-tuning on a portion of it triggers a pronounced post-tuning shift in the model's outputs; conversely, truly novel data elicits more modest changes. By comparing the model's output shifts for a suspicious dataset against those for a known non-member validation set, we statistically determine whether the suspicious set is likely to have been part of the model's original training corpus. Extensive experiments on both open-source and API-based LLMs validate CatShift's effectiveness in logit-inaccessible settings, offering a robust and practical solution for safeguarding proprietary data.

Problem

• Detect unauthorized dataset usage in LLMs without log probabilities
• Address copyright risks from proprietary data in large language models
• Develop label-only method to identify training data membership

Innovation

• Label-only dataset inference without logits
• Utilizes catastrophic forgetting for detection
• Compares output shifts for statistical validation