Agentic AI Scientists Are Not Built For Autonomous Scientific Discovery

2026-05-09

AI Summary

Current AI systems fall short of achieving end-to-end autonomous scientific discovery due to structural bottlenecks, including bias in problem selection, absence of tacit experimental knowledge, compression of output diversity, and lack of physical feedback. This work systematically analyzes the fundamental limitations of existing architectures, arguing that these issues cannot be resolved solely through scaling or incremental engineering fixes, necessitating a paradigm shift. To this end, the study proposes four key innovations: employing scientific simulations as training verifiers, developing persistent world models capable of representing dynamic research objectives, establishing a pre-registered repository for AI-generated hypotheses, and prioritizing scientific needs over tool capabilities in system design. Collectively, these contributions offer a theoretical framework and actionable pathways toward realizing AI with genuine scientific agency.

Abstract

A growing body of work pursues AI scientists capable of end-to-end autonomous scientific discovery. This position paper argues that although they already function as co-scientists, agentic AI scientists are not built for autonomous scientific discovery. We identify the following challenges in building and deploying autonomous AI scientists: (1) Problem selection is influenced by the McNamara fallacy; (2) Agents are built on large language models (LLMs) whose training corpora omit tacit procedural and failure knowledge of laboratory practice; (3) Preference optimisation during post-training compresses output diversity toward consensus; and (4) Most scientific benchmarks measure single-turn prediction accuracy and lack feedback from physical experiments back to the computational model. These challenges are not just questions of scale and scaffolding; they require revisiting fundamental design choices. To build truly autonomous AI scientists, we recommend the use of scientific simulations as verifiers for training, the design of persistent world models that represent the shifting objectives governing real investigations, the establishment of a centralized preregistration repository for all AI-generated hypotheses, and application driven by scientific need rather than tool affordance.

Problem

• autonomous scientific discovery
• agentic AI
• large language models
• scientific benchmarks
• tacit knowledge

Innovation

• Agentic AI
• Autonomous Scientific Discovery
• Large Language Models
• Scientific Simulation
• Tacit Knowledge