Automated Adversarial Collaboration for Advancing Theory Building in the Cognitive Sciences

2026-04-28

AI Summary

Traditional approaches to evaluating cognitive theories are constrained by narrow experimental paradigms and pairwise model comparisons, hindering cross-task integration of evidence. This work proposes the first closed-loop, simulation-based adversarial collaboration framework that dynamically generates candidate models and validation protocols without predefined experimental conditions, enabling automated adjudication among competing cognitive theories. The framework integrates large language model–driven theoretical agents, program synthesis, and information-theoretic experimental design. In simulations involving three canonical categorization theories, the system successfully recovered ground-truth theories across multiple noise conditions, with only minor performance degradation under the most challenging settings, thereby demonstrating its effectiveness and robustness.

Abstract

Cognitive science often evaluates theories through narrow paradigms and local model comparisons, limiting the integration of evidence across tasks and realizations. We introduce an automated adversarial collaboration framework for adjudicating among competing theories even when the candidate models and experiments must be discovered during the adjudication process. The system combines LLM-based theory agents, program synthesis, and information-theoretic experimental design in a closed loop. In a simulation study spanning three classic categorization theories, the framework recovered the ground-truth theory across noise settings with weaker reliability in the hardest settings. Together, the framework and findings provide a concrete proof of concept for closed-loop, in-silico theory adjudication in cognitive science.

Problem

• theory adjudication
• cognitive science
• adversarial collaboration
• model comparison
• evidence integration

Innovation

• adversarial collaboration
• theory adjudication
• large language models
• program synthesis
• information-theoretic experimental design