Can we automatize scientific discovery in the cognitive sciences?

2026-03-21

AI Summary

This work proposes the first end-to-end automated framework for cognitive science discovery, addressing the limitations of traditional approaches that rely on manually designed experiments and models constrained by researchers’ intuitions. The framework leverages large language models (LLMs) to autonomously generate experimental paradigms, simulate high-fidelity behavioral data, and synthesize computational cognitive models. Crucially, it implements a closed-loop optimization process driven by LLM-based evaluations of theoretical “interestingness.” By fully automating the pipeline—from experimental design and model generation to theory assessment—this system overcomes human cognitive biases and scalability bottlenecks, establishing a high-throughput discovery engine capable of rapidly producing conceptually meaningful candidate theories that serve as high-quality hypotheses for subsequent empirical validation.

Technology Category

• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior
• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

The cognitive sciences aim to understand intelligence by formalizing underlying operations as computational models. Traditionally, this follows a cycle of discovery where researchers develop paradigms, collect data, and test predefined model classes. However, this manual pipeline is fundamentally constrained by the slow pace of human intervention and a search space limited by researchers' background and intuition. Here, we propose a paradigm shift toward a fully automated, in silico science of the mind that implements every stage of the discovery cycle using Large Language Models (LLMs). In this framework, experimental paradigms exploring conceptually meaningful task structures are directly sampled from an LLM. High-fidelity behavioral data are then simulated using foundation models of cognition. The tedious step of handcrafting cognitive models is replaced by LLM-based program synthesis, which performs a high-throughput search over a vast landscape of algorithmic hypotheses. Finally, the discovery loop is closed by optimizing for ''interestingness'', a metric of conceptual yield evaluated by an LLM-critic. By enabling a fast and scalable approach to theory development, this automated loop functions as a high-throughput in-silico discovery engine, surfacing informative experiments and mechanisms for subsequent validation in real human populations.

Problem

• scientific discovery
• cognitive sciences
• automation
• computational models
• Large Language Models

Innovation

• automated scientific discovery
• large language models
• cognitive modeling
• program synthesis
• in silico experimentation