The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Search

2025-04-10

AI Summary

Achieving a fully autonomous, end-to-end AI-driven scientific research loop—including hypothesis generation, experimental design and execution, data analysis/visualization, and scholarly writing—remains an open challenge due to reliance on manual code templates and limited cross-domain generalization. Method: We propose a progressive agent-tree search framework that eliminates template-based coding and enables generalization across machine learning domains; integrate a vision-language model (VLM)-driven chart review feedback loop to enhance result interpretability and credibility; and unify multi-agent collaboration, automated code generation & execution, and scientific writing & typesetting. Contribution/Results: Three fully AI-generated papers were submitted to ICLR workshops; one was accepted after formal peer review with a score exceeding the average acceptance threshold for human-authored submissions—the first such publication globally validated through rigorous academic peer review.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

AI is increasingly playing a pivotal role in transforming how scientific discoveries are made. We introduce The AI Scientist-v2, an end-to-end agentic system capable of producing the first entirely AI generated peer-review-accepted workshop paper. This system iteratively formulates scientific hypotheses, designs and executes experiments, analyzes and visualizes data, and autonomously authors scientific manuscripts. Compared to its predecessor (v1, Lu et al., 2024 arXiv:2408.06292), The AI Scientist-v2 eliminates the reliance on human-authored code templates, generalizes effectively across diverse machine learning domains, and leverages a novel progressive agentic tree-search methodology managed by a dedicated experiment manager agent. Additionally, we enhance the AI reviewer component by integrating a Vision-Language Model (VLM) feedback loop for iterative refinement of content and aesthetics of the figures. We evaluated The AI Scientist-v2 by submitting three fully autonomous manuscripts to a peer-reviewed ICLR workshop. Notably, one manuscript achieved high enough scores to exceed the average human acceptance threshold, marking the first instance of a fully AI-generated paper successfully navigating a peer review. This accomplishment highlights the growing capability of AI in conducting all aspects of scientific research. We anticipate that further advancements in autonomous scientific discovery technologies will profoundly impact human knowledge generation, enabling unprecedented scalability in research productivity and significantly accelerating scientific breakthroughs, greatly benefiting society at large. We have open-sourced the code at https://github.com/SakanaAI/AI-Scientist-v2 to foster the future development of this transformative technology. We also discuss the role of AI in science, including AI safety.

Problem

• Develops an AI system for fully autonomous scientific paper generation
• Eliminates human-authored code templates in machine learning research
• Enhances peer-review acceptance via agentic tree-search and VLM feedback

Innovation

• Agentic tree-search methodology for automation
• Vision-Language Model for feedback refinement
• End-to-end autonomous scientific manuscript generation