Is Data Shapley Not Better than Random in Data Selection? Ask NASH

2026-05-11

AI Summary

This work addresses the instability of traditional Data Shapley–based data selection methods, which in certain scenarios perform no better than random selection. To overcome this limitation, the authors propose the NASH framework, which for the first time identifies and leverages subtask structures that are “Shapley-informative.” By decomposing the target utility function into multiple Shapley-informative components and optimizing data selection through nonlinear aggregation, NASH significantly enhances both the stability and performance of Shapley-based approaches. Notably, this improvement is achieved with negligible additional computational overhead, and the method consistently outperforms existing baselines across a range of settings.

Abstract

Data selection studies the problem of identifying high-quality subsets of training data. While some existing works have considered selecting the subset of data with top-$m$ Data Shapley or other semivalues as they account for the interaction among every subset of data, other works argue that Data Shapley can sometimes perform ineffectively in practice and select subsets that are no better than random. This raises the questions: (I) Are there certain "Shapley-informative" settings where Data Shapley consistently works well? (II) Can we strategically utilize these settings to select high-quality subsets consistently and efficiently? In this paper, we propose a novel data selection framework, NASH (Non-linear Aggregation of SHapley-informative components), which (I) decomposes the target utility function (e.g., validation accuracy) into simpler, Shapley-informative component functions, and selects data by optimizing an objective that (II) aggregates these components non-linearly. We demonstrate that NASH substantially boosts the effectiveness of Shapley/semivalue-based data selection with minimal additional runtime cost.

Problem

• Data Selection
• Data Shapley
• Semivalues
• High-quality Subsets
• Shapley-informative

Innovation

• Data Shapley
• data selection
• NASH
• non-linear aggregation
• semivalue