GraphReview: Scientific Paper Evaluation via LLM-Based Graph Message Passing

2026-05-26

AI Summary

This work addresses the challenge that existing large language models struggle to consistently aggregate cross-paper evidence for peer review. The authors propose a novel approach that formulates review as a message-passing process over a semantic paper graph, integrating intrinsic paper quality, contemporaneous associations, and longitudinal citation relationships. Leveraging large language models, the method generates node-level quality priors and edge-level comparative evidence, which are then propagated via personalized PageRank to disseminate review signals. A reward-induced maximum likelihood objective is introduced to optimize the model, enabling effective generalization of review evidence across papers. Experiments demonstrate substantial improvements—averaging a 29.7% gain in overall performance (23.7% higher acceptance prediction accuracy and 57.6% increase in Spearman correlation)—alongside higher-quality review text generation, with consistent generalization across multiple conferences and time periods.

Abstract

Scientific paper evaluation often involves not only assessing a manuscript itself, but also relating it to contemporaneous research and prior literature. However, existing LLM-based methods typically model these signals separately and lack a unified mechanism for propagating review evidence across papers. We propose $\textbf{GraphReview}$, a graph-based LLM framework that formulates paper evaluation as review-signal message passing over a semantic paper graph. The graph jointly captures intrinsic quality, synchronic links among contemporaneous papers, and diachronic links to prior work. LLMs are used to estimate node-level quality priors and generate edge-level comparative evidence through pairwise paper comparisons, while Personalized PageRank integrates review signals for quality ranking, decision prediction, and review generation. To produce higher-quality graph evidence, we propose reward-induced maximum likelihood objectives for training the LLM backbones. Experiments show that GraphReview consistently outperforms the strongest baseline, achieving average improvements of 29.7% on decision and ranking metrics, including gains of 23.7% in Accuracy and 57.6% in Spearman's $ρ$. It also produces higher-quality review texts and generalizes effectively across time periods and conference venues. The code is available at https://github.com/ECNU-Text-Computing/GraphReview.

Problem

• scientific paper evaluation
• LLM-based review
• graph message passing
• review evidence propagation
• paper quality assessment

Innovation

• Graph-based LLM
• Message Passing
• Scientific Paper Evaluation
• Personalized PageRank
• Reward-Induced Training