Scaling Generalist Data-Analytic Agents

2025-09-29

AI Summary

Existing open-source data agents exhibit limited performance on multi-format, large-scale, and long-horizon multi-step data analysis tasks, primarily due to data scarcity and suboptimal training strategies. This paper introduces DataMind, a general-purpose data parsing agent that innovatively designs a fine-grained task taxonomy and progressive task composition mechanism; proposes knowledge-augmented trajectory sampling and a dynamic SFT+RL hybrid training objective; and implements a low-memory, high-stability multi-turn code execution framework. Trained on our self-constructed high-quality synthetic dataset DataMind-12K, the resulting DataMind-14B and DataMind-7B models achieve average scores of 71.16% and 68.10%, respectively, across multiple data analysis benchmarks—outperforming DeepSeek-V3.1 and GPT-5, and establishing new state-of-the-art results among open-source models. All datasets, models, and code will be publicly released.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Data Mining & Knowledge Management: Linked Open Data, Knowledge Graphs & KB Completion
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Web Mining and Content Analysis: Large pretrained models with web data
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Data-analytic agents are emerging as a key catalyst for automated scientific discovery and for the vision of Innovating AI. Current approaches, however, rely heavily on prompt engineering over proprietary models, while open-source models struggle to face diverse-format, large-scale data files and long-horizon, multi-step reasoning that real-world analytics demands. This paper introduces DataMind, a scalable data synthesis and agent training recipe designed to build generalist data-analytic agents. DataMind tackles three key challenges in building open-source data-analytic agents, including insufficient data resources, improper training strategy, and unstable code-based multi-turn rollout. Concretely, DataMind applies 1) a fine-grained task taxonomy and a recursive easy-to-hard task composition mechanism to increase the diversity and difficulty of synthesized queries; 2) a knowledge-augmented trajectory sampling strategy followed by model-based and rule-based filtering; 3) a dynamically adjustable training objective combining both SFT and RL losses; 4) a memory-frugal and stable code-based multi-turn rollout framework. Built on DataMind, we curate DataMind-12K, a high-quality trajectory set spanning diverse domains, task categories, and data file formats for data-analytic tasks. Trained on DataMind-12K, our DataMind-14B achieves state-of-the-art with an average score of 71.16% on multiple data analysis benchmarks, outperforming the strongest proprietary baselines DeepSeek-V3.1 and GPT-5. Our DataMind-7B also performs best among all open-source models with a score of 68.10%. We also incorporate some empirical insights gained from our exploratory trials into the analysis experiments, aiming to provide actionable insights about agentic training for the community. We will release DataMind-12K and DataMind-7B,14B for the community's future research.

Problem

• Open-source models struggle with diverse-format large-scale data files
• Current approaches lack proper training strategies for data-analytic agents
• Existing methods face unstable code-based multi-turn reasoning challenges

Innovation

• Fine-grained task taxonomy with recursive composition
• Knowledge-augmented trajectory sampling with filtering
• Dynamic training objective combining SFT and RL