AutoSynthesis: An agentic system for automated meta-analysis

2026-07-16

AI Summary

This study addresses the scalability limitations of quantitative evidence synthesis, which remains heavily reliant on manual effort and thus impedes timely access to reliable knowledge in research, medicine, and policy. The authors propose the first end-to-end multi-agent system capable of autonomously conducting full meta-analyses from natural language research questions. The system performs literature retrieval, screening, data extraction, effect size computation, and random-effects meta-analysis, while also supporting heterogeneity assessment and risk-of-bias evaluation. It generates complete, PRISMA-compliant meta-analysis reports without human intervention. Validated on over 28 studies, the system extracted more than 20 quantitative findings and produced pooled effect estimates highly concordant with those derived manually by experts, substantially enhancing the scalability, transparency, and reliability of evidence synthesis.

Abstract

Evidence synthesis is crucial for turning primary research into reliable knowledge for science, medicine, education, and policy. Yet, quantitative evidence synthesis remains largely manual and difficult to scale. Here, we introduce AutoSynthesis, an end-to-end multi-agent system for automated meta-analysis. Given a research question in natural language, AutoSynthesis formulates a search strategy, retrieves scientific literature, screens candidate studies, assesses full-text eligibility, extracts quantitative statistics, computes standardized effect sizes, and finally performs random-effects meta-analysis. AutoSynthesis further supports heterogeneity analysis to examine how effect sizes vary across moderators, as well as risk-of-bias assessment. As output, AutoSynthesis produces a transparent report aligned with PRISMA guidelines. In our application, AutoSynthesis screened over 28 studies and extracted more than 20 quantitative claims. The pooled effect estimates produced by AutoSynthesis are similar to Hedges' $g$ of expert-conducted meta-analyses, indicating close agreement with manual evidence synthesis. Together, these results show that AutoSynthesis can make quantitative evidence synthesis more scalable, thereby supporting evidence-based decision-making across disciplines.

Problem

• evidence synthesis
• meta-analysis
• automation
• scalability
• quantitative evidence

Innovation

• automated meta-analysis
• multi-agent system
• evidence synthesis
• effect size estimation
• PRISMA-compliant reporting