How Far Are AI Scientists from Changing the World?

2025-07-31

AI Summary

Current AI scientist systems—despite rapid advances in large language models—remain far from driving transformative scientific paradigm shifts. Key limitations include insufficient causal reasoning, fragmented cross-domain knowledge integration, lack of experimental closed-loop validation, poor interpretability, shallow domain expertise, and absence of rigorous verification mechanisms. Method: This paper proposes a forward-looking survey framework to systematically assess the capabilities and boundaries of LLM-based AI scientist systems across three core dimensions: automated scientific discovery, knowledge generation, and research process modeling. Contribution/Results: We identify essential components required for breakthrough discoveries and pinpoint critical bottlenecks impeding major scientific advances. Based on this analysis, we delineate a staged evolutionary pathway toward “world-class AI scientists,” specifying concrete intermediate objectives and milestones. The study provides both a theoretical foundation and a practical roadmap for AI-augmented fundamental scientific research.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Humans and AI: Other Foundations of Human Computation & AI
• Cognitive Modeling & Cognitive Systems: Agent Architectures

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems

Abstract

The emergence of large language models (LLMs) is propelling automated scientific discovery to the next level, with LLM-based Artificial Intelligence (AI) Scientist systems now taking the lead in scientific research. Several influential works have already appeared in the field of AI Scientist systems, with AI-generated research papers having been accepted at the ICLR 2025 workshop, suggesting that a human-level AI Scientist capable of uncovering phenomena previously unknown to humans, may soon become a reality. In this survey, we focus on the central question: How far are AI scientists from changing the world and reshaping the scientific research paradigm? To answer this question, we provide a prospect-driven review that comprehensively analyzes the current achievements of AI Scientist systems, identifying key bottlenecks and the critical components required for the emergence of a scientific agent capable of producing ground-breaking discoveries that solve grand challenges. We hope this survey will contribute to a clearer understanding of limitations of current AI Scientist systems, showing where we are, what is missing, and what the ultimate goals for scientific AI should be.

Problem

• Assessing AI Scientists' potential to revolutionize scientific research
• Identifying key bottlenecks in current AI Scientist systems
• Defining ultimate goals for scientific AI breakthroughs

Innovation

• LLM-based AI Scientist systems lead research
• AI-generated papers accepted at ICLR 2025
• Survey identifies bottlenecks for groundbreaking discoveries