AI scientists produce results without reasoning scientifically

2026-04-20

AI Summary

It remains unclear whether current large language model–driven scientific agents possess reasoning capabilities aligned with scientific epistemic norms. This study conducts over 25,000 agent experiments across eight research scenarios, integrating variance decomposition and cognitive-behavioral modeling to systematically analyze their reasoning mechanisms from a normative cognitive perspective for the first time. The findings reveal that base models predominantly determine agent performance and behavior, accounting for 41.4% of observed variance. Moreover, 68% of reasoning trajectories disregard empirical evidence, and only 26% demonstrate the capacity to revise beliefs in response to counter-evidence. These results indicate that existing agents lack genuine scientific reasoning abilities, and external scaffolding frameworks are insufficient to compensate for their intrinsic cognitive deficiencies.

Technology Category

• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty
• Knowledge Representation and Reasoning: Reasoning with Beliefs

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Agentic search
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Large language model (LLM)-based systems are increasingly deployed to conduct scientific research autonomously, yet whether their reasoning adheres to the epistemic norms that make scientific inquiry self-correcting is poorly understood. Here, we evaluate LLM-based scientific agents across eight domains, spanning workflow execution to hypothesis-driven inquiry, through more than 25,000 agent runs and two complementary lenses: (i) a systematic performance analysis that decomposes the contributions of the base model and the agent scaffold, and (ii) a behavioral analysis of the epistemological structure of agent reasoning. We observe that the base model is the primary determinant of both performance and behavior, accounting for 41.4% of explained variance versus 1.5% for the scaffold. Across all configurations, evidence is ignored in 68% of traces, refutation-driven belief revision occurs in 26%, and convergent multi-test evidence is rare. The same reasoning pattern appears whether the agent executes a computational workflow or conducts hypothesis-driven inquiry. They persist even when agents receive near-complete successful reasoning trajectories as context, and the resulting unreliability compounds across repeated trials in epistemically demanding domains. Thus, current LLM-based agents execute scientific workflows but do not exhibit the epistemic patterns that characterize scientific reasoning. Outcome-based evaluation cannot detect these failures, and scaffold engineering alone cannot repair them. Until reasoning itself becomes a training target, the scientific knowledge produced by such agents cannot be justified by the process that generated it.

Problem

• scientific reasoning
• large language models
• epistemic norms
• autonomous scientific agents
• reasoning reliability

Innovation

• scientific reasoning
• large language models
• epistemic norms
• agent scaffolding
• belief revision