Train on Validation (ToV): Fast data selection with applications to fine-tuning

2025-09-30

AI Summary

Data selection for fine-tuning under scarce target-distribution samples remains challenging. Method: This paper proposes a “validation-set-driven data selection” paradigm: it swaps the conventional roles of validation set and training pool—performing lightweight fine-tuning on the validation set and selecting the most discriminative samples from the training pool based on the magnitude of prediction shifts induced by fine-tuning. The method requires no additional annotations or gradient computations, ensuring both efficiency and theoretical interpretability. Results: Evaluated on instruction tuning and named entity recognition, it significantly reduces test log-loss on the target distribution, consistently outperforming existing SOTA methods on average while improving data utilization efficiency and fine-tuning performance. Its core innovation lies in the first use of the validation set as a proxy for fine-tuning and leveraging prediction shift as the selection criterion—enabling precise identification of high-information samples under few-shot settings.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Search and Optimization: Learning to Search
• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

State-of-the-art machine learning often follows a two-stage process: $(i)$~pre-training on large, general-purpose datasets; $(ii)$~fine-tuning on task-specific data. In fine-tuning, selecting training examples that closely reflect the target distribution is crucial. However, it is often the case that only a few samples are available from the target distribution. Existing data selection methods treat these target samples as a validation set and estimate the effect of adding or removing a single sample from the training pool by performing inference on the validation set. We propose a simpler and faster alternative that inverts the usual role of train and validation: we perform inference on the training pool before and after fine-tuning on the validation set. We then select samples whose predictions change the most. Our key insight is that the training samples most affected by fine-tuning on a small validation set tend to be the most beneficial for reducing test loss on the target distribution. Experiments on instruction tuning and named entity recognition tasks show that, in most cases, our method achieves lower test log-loss than state-of-the-art approaches. We support our findings with theoretical analysis.

Problem

• Selecting optimal training samples for fine-tuning with limited target data
• Improving data selection efficiency by reversing train-validation roles
• Reducing test loss through samples most affected by validation fine-tuning

Innovation

• Inverts train-validation roles for data selection
• Selects samples with most prediction changes after fine-tuning
• Uses small validation set to identify beneficial training samples