Convex Dataset Valuation for Post-Training

2026-05-15

AI Summary

This work addresses the challenge of efficiently selecting auxiliary data subsets to enhance target task performance in large language model post-training, where computational, annotation, and licensing costs impose strict budget constraints. The problem is formulated as a budget-constrained subset selection task, and the paper introduces the first scalable convex optimization–based dataset valuation method. This approach leverages kernel mean matching (KMM) in gradient space to jointly assess both the alignment between the candidate data and the target task and the internal redundancy within the data subset. Unlike existing metrics that consider only gradient alignment, the proposed method explicitly models redundancy, making it particularly suitable for budget-aware scenarios such as data markets. Experiments demonstrate that the method significantly outperforms current baselines across diverse post-training settings while maintaining low computational overhead, thereby effectively improving target task performance.

Abstract

Improving LLM performance on downstream tasks sometimes requires leveraging auxiliary datasets during post-training. In practice, however, developers face constraints on compute, labeling, and licensing costs that preclude using all available data, necessitating principled dataset-level selection. These constraints are increasingly shaped by dataset marketplaces, where data acquisition is governed by budgets and negotiation. We study dataset valuation as a subset selection problem during LLM post-training. Our goal is to identify and weight auxiliary datasets so as to maximize target task performance given constrained budgets. We first show that commonly used gradient alignment scores provide a reasonable yet incomplete valuation signal, as they ignore redundancy among datasets. To address this, we propose a scalable convex dataset-level valuation method based on kernel mean matching (KMM) in gradient space, which jointly accounts for alignment with the target task and redundancy across auxiliary datasets. Through extensive experiments across diverse post-training settings and tasks, we show that our approach consistently outperforms existing valuation baselines, achieving stronger performance with low computational overhead. Our results position dataset valuation as a practical decision tool for post-training data selection in market-constrained large language model settings. The code is available at https://github.com/uiuctml/convex_data_valuation.

Problem

• dataset valuation
• post-training
• large language models
• budget-constrained selection
• subset selection

Innovation

• convex dataset valuation
• kernel mean matching
• post-training
• dataset selection
• large language models