DIPBox: A Multi-scale Testing Framework for Tracking Dataset Regeneration

2026-06-19

AI Summary

This work addresses the challenge of tracing unauthorized replication and regeneration of training datasets, a task where existing methods struggle to reliably detect both full and partial data regeneration. To overcome this limitation, the authors propose DIPBox, a multi-scale testing framework that constructs four similarity metrics based on sample-level, set-level, and distribution-level features to identify suspicious regeneration under varying degrees of information access. DIPBox is the first method to enable multi-scale tracing of unknown adversarial data regeneration, revealing an inherent trade-off between model utility and detectability while providing theoretical guarantees. Extensive experiments across 16 base datasets, 320 regenerated variants, and 590 derived models demonstrate that DIPBox significantly outperforms current approaches and exhibits strong robustness against three types of adaptive attacks.

Abstract

Training datasets have tremendous proprietary value and are vulnerable to unauthorized copying. Existing defenses mainly focus on tracking individual data points, but pay little attention to the threat of dataset regeneration. Through a measurement study of public tumor datasets, we identify substantial real-world partial-dataset replication, raising concerns about potential license noncompliance. To counter the challenge of tracking previously unknown adversarial regeneration, our key insight is that regeneration that preserves model utility inevitably preserves measurable signals across multiple feature scales. We categorize these dataset features into sample-, set-, and distribution-level features and design four similarity metrics to accurately identify regeneration. Based on these metrics, we develop DIPBox, which to our knowledge is the first testing framework that tracks regeneration suspects via multi-scale similarity testing across a spectrum of defender access settings, from limited to full information. We further provide a learning-theoretic analysis that justifies these multi-scale metrics and formalizes an inherent utility--divergence trade-off, implying fundamental limits on evasive regeneration. Extensive experiments on 16 vision and text base datasets, 320 regenerated datasets, and 590 derived models validate that DIPBox outperforms previous solutions while characterizing its robustness and limits under three adaptive attacks.

Problem

• dataset regeneration
• data copying
• license compliance
• data tracking
• adversarial regeneration

Innovation

• dataset regeneration
• multi-scale similarity
• DIPBox
• utility-divergence trade-off
• data provenance