Building Agent Harnesses for Scientific Curation from Multimodal Sources

2026-06-18

AI Summary

This work addresses the challenge that critical evidence in scientific literature is scattered across lengthy texts, tables, and figures, hindering existing agents from efficiently performing cross-modal structured extraction and reasoning. To overcome this limitation, the authors propose Beaver, a novel framework that integrates task scaffolding, multimodal evidence tools, and provenance tracking into an agent workflow, enabling auditable, phased autonomous research with iterative diagnose-and-correct cycles. Evaluated on the Gold-Referenced Attribute Score (GRAS), Beaver achieves 81.0—surpassing the current state-of-the-art agent by 23 percentage points—with particularly pronounced gains on high-value attributes requiring cross-modal reasoning.

Abstract

Scientific discovery workflows often depend on structured curation from the literature. This is difficult for current agents because the key evidence is scattered across long text, dense tables, and figures, and the final records often require reasoning across multiple evidence fragments rather than copying a single span. We study scientific curation from multimodal sources and introduce Beaver, an agent harness that extracts structured information from scientific papers while preserving provenance to the supporting evidence. Beaver combines a frontier agent with multimodal evidence tooling, task scaffolding, and artifact-grounded autoresearch. These components turn curation into a staged, auditable workflow and enable an iterative evaluate--diagnose--revise loop, where persistent run artifacts expose stage-localized failures and guide harness updates. Experiments show that Beaver reaches 81.0 on Gold-Referenced Attribute Score (GRAS), an attribute-level measure of agreement with gold curated records, outperforming frontier agents by over 23 absolute points. Ablations show that task scaffolding, multimodal evidence tooling, and provenance traces each contribute meaningfully to performance, while attribute-level analysis shows the largest gains on high-value attributes that require cross-modal reasoning and normalization. These results show that, for scientific curation from papers with multimodal evidence, harness design is a central determinant of agent performance.

Problem

• scientific curation
• multimodal sources
• structured information extraction
• cross-modal reasoning
• evidence integration

Innovation

• multimodal scientific curation
• agent harness
• task scaffolding
• provenance tracing
• artifact-grounded autoresearch