Provenance Before Prose: Claim-Locked Reporting

2026-08-25

AI Summary

研究解决了大语言模型在生成统计报告时可能出现的数据漂移、效果方向反转等问题，通过提出并验证了一种先确定证据来源和数据再生成文本的‘claim-locked reporting’方法来提高报告的准确性和可重复性。

Abstract

Large language models (LLMs) can fluently verbalize statistical evidence, yet statistical reports can still drift numerical values, invert effect directions, or restate thresholded contrasts as categorical effects. We frame these failures as a control problem: the evidence-bearing content of a scientific report should be fixed by structured statistical results rather than sampled during prose generation. We therefore use cross-run reproducibility to stress-test whether report-visible numbers and claims are bound before prose generation. Existing controls operate at the text or slot level; a deterministic hybrid template reproduces only 61.1% of report-visible numerical content across seeds because the LLM still selects which findings and numbers the template renders. We propose claim-locked reporting, a provenance-before-prose protocol that fixes the evidence source, numbers, direction, and allowed language strength of each reportable claim before the LLM writes only connective prose. Across fMRI functional-connectivity reporting and randomized controlled trial reporting on Evidence Inference 2.0, claim-locked reporting improves reproducibility over the hybrid template by 37.4 and 20.5 points, respectively. Blinded human audits support the observed direction-preservation and governance trends. In an fMRI cost analysis with DeepSeek, claim-locked reporting also yields the lowest observed token use and median generation latency.

Problem

• Large language models
• statistical evidence
• reporting errors
• control problem
• reproducibility

Innovation

• claim-locked reporting
• reproducibility
• statistical evidence
• large language models
• numerical content