Problem
Research questions and friction points this paper is trying to address.
instruction data selection
manual heuristic design
data curation
generalization across domains
user intent interpretation
Agentic Instruction Data Selection: Let DataMaster Interpret Your Intent
🤖 AI Summary
Existing instruction data selection methods rely on single metrics or handcrafted heuristics, which struggle to adapt to the complexity and variability of real-world scenarios, often resulting in inefficiency and errors. This work proposes DataMaster—the first large language model–based agent for instruction data selection—that interprets user intent through natural language understanding and autonomously orchestrates and executes optimal selection strategies, thereby shifting the paradigm from manual configuration to automated, personalized collaboration. Experimental results demonstrate that DataMaster consistently outperforms static baselines across diverse domains such as mathematics, healthcare, and code generation, and in many settings even surpasses models trained on full datasets, significantly enhancing both selection flexibility and downstream performance.
📝 Abstract
Although existing instruction data selection methods have introduced various metrics, the inherent complexity of real-world datasets makes it impractical for any single metric to generalize across all scenarios. Developers are thus often forced to manually inspect data and craft heuristic rules for each new application---a tedious and error-prone process. In this paper, we propose a paradigm shift from manual configuration to automated orchestration via the Instruction Data Selection Agent (DataMaster), which interprets user intent and autonomously composes optimal selection strategies. By allowing users to specify data needs through natural language descriptions, DataMaster simplifies data curation and removes the burden of manual strategy design. Extensive experiments across the math, medical, and code domains show that DataMaster outperforms static baselines in most settings and surpasses full-pool training in a substantial number of cases. The implementation of DataMaster and the scripts needed to reproduce the reported pipeline are publicly available at https://github.com/nju-websoft/DataMaster.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Agentic AI
Instruction Data Selection
Natural Language Intent
Automated Data Curation
DataMaster
🔎 Similar Papers
No similar papers found.
