Can AI Be a Good Peer Reviewer? A Survey of Peer Review Process, Evaluation, and the Future

2026-04-30

AI Summary

This study systematically investigates the application of artificial intelligence—particularly large language models—across the entire academic peer review pipeline, encompassing key stages such as review generation, author rebuttal, meta-reviewing, and manuscript revision. The work introduces the first end-to-end technical framework that integrates fine-tuning, agent-based systems, reinforcement learning, and multidimensional automated evaluation methods, comprehensively mapping existing datasets, modeling paradigms, and assessment strategies. Beyond offering practical guidelines for building end-to-end peer review assistance systems, the research also critically examines associated ethical challenges and outlines promising future directions, thereby providing both theoretical grounding and actionable insights for advancing AI-supported scholarly peer review.

Abstract

Peer review is a multi-stage process involving reviews, rebuttals, meta-reviews, final decisions, and subsequent manuscript revisions. Recent advances in large language models (LLMs) have motivated methods that assist or automate different stages of this pipeline. In this survey, we synthesize techniques for (i) peer review generation, including fine-tuning strategies, agent-based systems, RL-based methods, and emerging paradigms to enhance generation; (ii) after-review tasks including rebuttals, meta-review and revision aligned to reviews; and (iii) evaluation methods spanning human-centered, reference-based, LLM-based and aspect-oriented. We catalog datasets, compare modeling choices, and discuss limitations, ethical concerns, and future directions. The survey aims to provide practical guidance for building, evaluating, and integrating LLM systems across the full peer review workflow.

Problem

• peer review
• large language models
• AI evaluation
• scientific publishing
• review automation

Innovation

• large language models
• peer review automation
• review generation
• rebuttal and meta-review
• evaluation frameworks