Autodata: An agentic data scientist to create high quality synthetic data

2026-06-24

AI Summary

This work proposes an innovative framework that addresses the longstanding challenge of efficiently generating high-quality synthetic data for training and evaluation. By introducing AI agents into the data generation pipeline, the approach uniquely combines agentic self-instruction with meta-optimization, enabling agents to assume the role of data scientists who iteratively refine synthetic data quality. This paradigm effectively converts reasoning compute into measurable data quality improvements. Empirical results demonstrate that the method substantially outperforms existing synthetic data generation techniques across diverse domains—including computer science, legal reasoning, and mathematical reasoning—with performance further enhanced through meta-optimization.

Abstract

We introduce Autodata, a general method that enables AI agents to act as data scientists who build high quality training and evaluation data. We show how to train (meta-optimize) such a data scientist agent, so that it learns to create even stronger data. We describe the overall formulation, and a specific practical implementation, Agentic Self-Instruct. We conduct experiments on computer science research tasks, legal reasoning tasks and reasoning with mathematical objects, where we obtain improved results compared to classical synthetic dataset creation methods. Further, meta-optimizing the data scientist agent itself delivers an even larger performance uplift. Agentic data creation provides a way to convert increased inference compute into higher quality model training. Overall, we believe this direction has the potential to change the way we build AI data.

Problem

• synthetic data
• data creation
• AI training
• high-quality data
• automated data generation

Innovation

• agentic data creation
• synthetic data
• meta-optimization
• AI agent
• self-instruct