Judgment-Grounded Expansion for Peer Review Generation

2026-06-22

AI Summary

Fully automated peer review lacks accountability and is ill-suited for contexts requiring human oversight. This work proposes a “judgment-guided expansion” paradigm, wherein human reviewers provide evaluative claims that the system expands into complete review texts through a structured generate–verify–refine pipeline. We formalize this task for the first time, establishing a collaborative review framework that balances automation with accountability. To support robust evaluation, we introduce candidate-set calibration and a large-scale assessment methodology grounded in conformal prediction. Experimental results demonstrate that conformal prediction effectively balances candidate set size against target coverage, offering both methodological foundations and empirical validation for building trustworthy collaborative peer review systems.

Abstract

Automatic review generation is a promising direction for accelerating scientific progress. While most work adopts an end-to-end setup, its fully automated nature makes it less suitable for settings that demand accountability. To better balance automation and accountability, we formalize judgment-grounded expansion, a human-AI collaboration mode where a reviewer provides an evaluative claim and the system expands it into review comment candidate(s). We model it as a structured generate-check-refine process and conduct a user study to collect human-model interaction data. We study two practical challenges for judgment-grounded expansion: scalable evaluation and candidate set curation. We develop methods to simulate the process for large-scale evaluation, and show that conformal prediction is well suited to balancing candidate set size and target coverage. Our work establishes judgment-grounded expansion as a concrete task and provides empirical and methodological foundations for the design of future collaborative review generation systems.

Problem

• peer review generation
• human-AI collaboration
• judgment-grounded expansion
• accountability
• candidate set curation

Innovation

• judgment-grounded expansion
• human-AI collaboration
• peer review generation
• conformal prediction
• structured generate-check-refine