AIBuildAI-2: A Knowledge-Enhanced Agent for Automatically Building AI Models

2026-05-26

AI Summary

This work addresses the challenge faced by natural scientists who lack AI engineering expertise and struggle to efficiently develop high-performance models. To bridge this gap, the authors propose an intelligent agent equipped with a dynamic external knowledge system that automates model design and training. The system hierarchically organizes and loads AI development knowledge on demand, integrating large language models, knowledge retrieval, experience distillation, and structured knowledge back-writing. Its core innovation lies in constructing an evolvable, verifiable, and task-specific engineering knowledge base capable of continuous learning from real-world practice. Evaluated on MLE-Bench, the approach achieves a state-of-the-art medal rate of 70.7%, and ranks within the top 6.6% among 4,370 global teams in a competitive heart disease prediction challenge.

Abstract

AI models underpin data-centric applications from image and text processing to scientific discovery in biology, physics, and chemistry. Yet developing them remains heavily manual, requiring practitioners to design architectures, build training pipelines, and iteratively refine solutions, making it challenging for natural scientists without specialized AI engineering expertise to build the high-performing models their research demands. To reduce this burden and broaden access to AI for scientific discovery, agents that automatically build AI models have been proposed. However, the performance of these agents is largely limited by the parametric knowledge of their underlying large language models, which is static, often outdated, and sparse on practical AI model engineering know-how. To address this limitation, we introduce AIBuildAI-2, a knowledge-enhanced agent with an external, evolving knowledge system for automatically building AI models. The knowledge system of AIBuildAI-2 is hierarchical, organizing curated AI development knowledge into high-level knowledge instructions over topical categories and low-level knowledge documents under each category, from which the agent dynamically loads only the context relevant to its current state and the AI task being solved, grounding each design and implementation decision in concrete, externally verifiable expertise. The system is initialized by collecting and cleaning AI-development-related documents from the web and organizing them into the corresponding categories, and continually evolves from the agent's own experience by distilling each completed run on an AI task into structured takeaways that are written back into the knowledge system. AIBuildAI-2 achieves state-of-the-art results, ranking first on MLE-Bench with a 70.7% medal rate and placing in the top 6.6% among 4,370 human-expert teams in a heart disease prediction competition.

Problem

• AI model building
• knowledge limitation
• scientific discovery
• automated machine learning
• large language models

Innovation

• knowledge-enhanced agent
• automated AI model building
• evolving knowledge system
• dynamic knowledge retrieval
• hierarchical knowledge organization