Human-AI Collaboration in Science at Scale: A Global Large-scale Randomized Field Experiment

2026-05-22

AI Summary

Scientific feedback, a cornerstone of knowledge production, has long been constrained by limited accessibility, uneven distribution, and challenges in scaling. This study addresses these limitations through a large-scale global randomized controlled trial, leveraging large language models to generate domain-specific feedback for over 31,000 arXiv preprints, reaching more than 45,000 researchers across 150 disciplines and 133 countries. The work provides the first causal evidence that structured AI interventions can transform feedback from a scarce, private resource into an inclusive, shared tool, significantly increasing the likelihood of manuscript revision by authors (+12.55%) and fostering subsequent adoption of LLM-based tools. Notably, researchers from non-English-speaking regions, low-impact institutions, and early-career scholars derived the greatest benefits from this intervention.

Abstract

Collaboration is the defining mode of modern science, yet its core mechanism -- feedback -- remains hard to observe, difficult to scale, and unequally distributed. Here we test whether large language models (LLMs) can contribute to this hidden but vital practice and reallocate scientific feedback, an essential yet scarce resource for knowledge production. In a global large-scale randomized field experiment, we delivered customized LLM-generated feedback for over 31,000 arXiv preprints across 150 fields and more than 45,000 researchers from 133 geographic regions. Relative to controls, authors who received feedback had a significantly higher likelihood of revising their manuscripts, corresponding to a 12.55% relative increase over the baseline revision rate. Exposure to AI feedback also increased authors' subsequent use of LLM tools in their future papers, suggesting longer-run shifts in scientific practice. These effects were strongest among authors from non-English-dominant research regions, manuscripts less embedded in the scholarly literature, and teams with lower h-indexes and earlier career stages, consistent with the idea that AI feedback may provide the greatest benefit where access to timely critique is otherwise limited. Together, these findings provide causal evidence that structured AI-based interventions can transform access to scientific feedback from a largely private advantage into a more widely distributed resource, with broader implications for productivity, equity, and capacity across the global research system.

Problem

• scientific feedback
• human-AI collaboration
• research equity
• knowledge production
• feedback distribution

Innovation

• large language models
• scientific feedback
• randomized field experiment
• human-AI collaboration
• research equity