AutoMind: Adaptive Knowledgeable Agent for Automated Data Science

2025-06-12

AI Summary

Current LLM-driven data science agents exhibit limited performance in automating complex, innovative machine learning tasks due to rigid pipelines and insufficient modeling of domain expertise. This paper introduces ADSA (Adaptive Data Science Agent), a novel framework for automated data science. ADSA addresses these limitations through three core innovations: (1) expert knowledge base–augmented experience modeling and dynamic reuse; (2) an informed tree search algorithm enabling structured, interpretable reasoning navigation; and (3) task-complexity–aware adaptive code generation. The method integrates domain knowledge retrieval, hierarchical tree-based reasoning, collaborative LLM execution, and dynamic code optimization. Evaluated on two comprehensive benchmarks—AutoML-Bench and DSBench—ADSA achieves state-of-the-art performance across all metrics, significantly improving solution quality, execution efficiency, and generalization robustness.

Technology Category

• Multiagent Systems: Adversarial Agents
• Machine Learning: Auto ML and Hyperparameter Tuning
• Search and Optimization: Learning to Search

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling

Abstract

Large Language Model (LLM) agents have shown great potential in addressing real-world data science problems. LLM-driven data science agents promise to automate the entire machine learning pipeline, yet their real-world effectiveness remains limited. Existing frameworks depend on rigid, pre-defined workflows and inflexible coding strategies; consequently, they excel only on relatively simple, classical problems and fail to capture the empirical expertise that human practitioners bring to complex, innovative tasks. In this work, we introduce AutoMind, an adaptive, knowledgeable LLM-agent framework that overcomes these deficiencies through three key advances: (1) a curated expert knowledge base that grounds the agent in domain expert knowledge, (2) an agentic knowledgeable tree search algorithm that strategically explores possible solutions, and (3) a self-adaptive coding strategy that dynamically tailors code generation to task complexity. Evaluations on two automated data science benchmarks demonstrate that AutoMind delivers superior performance versus state-of-the-art baselines. Additional analyses confirm favorable effectiveness, efficiency, and qualitative solution quality, highlighting AutoMind as an efficient and robust step toward fully automated data science.

Problem

• Overcoming rigid workflows in LLM-driven data science agents
• Enhancing adaptability in automated machine learning pipelines
• Capturing empirical expertise for complex, innovative tasks

Innovation

• Curated expert knowledge base for domain expertise
• Agentic knowledgeable tree search for solution exploration
• Self-adaptive coding strategy for dynamic complexity handling