Online Data Selection for Instruction Tuning via Gaussian Processes

2026-06-29

AI Summary

This work addresses the limitation of existing online data selection methods for instruction fine-tuning of large language models, which rely on local utility within individual batches and struggle to identify globally high-quality samples. To overcome this, the authors propose GAIA, a novel framework that introduces Gaussian process regression into online data selection, modeling data value as a global continuous utility manifold in semantic space. GAIA integrates an adaptive strategy fusion mechanism with the Fixed-Share Hedge algorithm, providing theoretical guarantees of dynamic regret bounds under non-stationary data quality scores. Experimental results demonstrate that GAIA significantly outperforms state-of-the-art methods such as GREATS across three benchmark datasets, exhibiting superior efficiency, scalability, and robustness.

Abstract

With Large Language Model (LLM) pre-training and fine-tuning shifting its focus from data volume to data quality, quality data selection has emerged as a critical research topic. Existing online data selection methods for LLM training are typically "batch-constrained", limiting optimization to local utility within random batches. To overcome this, we propose GAIA (Global Adaptive Instruction tuning via GAussian processes), a framework that formulates data valuation as a global estimation process. GAIA employs Gaussian Process regression to model continuous utility manifolds across the semantic space, utilizing an adaptive strategy fusion mechanism to dynamically prioritize high-utility samples. By casting the strategy-posterior update as an instance of the classical fixed-share Hedge framework for tracking the best expert, we inherit a dynamic-regret guarantee that characterizes GAIA's robustness under non-stationary quality scores during training. Empirical evaluations on three datasets demonstrate that GAIA significantly outperforms state-of-the-art baselines like \greats, establishing our method as a scalable and robust solution for efficient instruction tuning.

Problem

• online data selection
• instruction tuning
• data quality
• large language models
• non-stationary utility

Innovation

• Gaussian Processes
• Online Data Selection
• Instruction Tuning
• Dynamic Regret
• Adaptive Strategy Fusion