🤖 AI Summary
Existing Shapley value–based data pruning methods struggle to effectively retain critical samples in low-data regimes. This work proposes Constrained Data Value Maximization (CDVM), which introduces constrained optimization into data value–driven pruning for the first time. By formulating the problem as a constrained optimization task, CDVM simultaneously maximizes the overall influence of the retained dataset while limiting the excessive contribution of any individual test sample, thereby achieving a balance between global impact and local fidelity. Experiments on the OpenDataVal benchmark demonstrate that CDVM significantly outperforms existing approaches when retaining only a small fraction of data, offering both superior performance and computational efficiency.
📝 Abstract
Attributing model behavior to training data is an evolving research field. A common benchmark is data removal, which involves eliminating data instances with either low or high values, then assessing a model's performance trained on the modified dataset. Many existing studies leverage Shapley-based data values for this task. In this paper, we demonstrate that these data values are not optimally suited for pruning low-value data when only a limited amount of data remains. To address this limitation, we introduce the Constraint-Data-Value-Maximization (CDVM) approach, which effectively utilizes data attributions for pruning in low-data scenarios. By casting pruning as a constrained optimization that both maximizes total influence and penalizes excessive per-test contributions, CDVM delivers robust performance when only a small fraction of the data is retained. On the OpenDataVal benchmark, CDVM shows strong performance and competitive runtime.