Reviewing the Reviewer: Elevating Peer Review Quality through LLM-Guided Feedback

2026-01-17

arXiv.org

AI Summary

本文针对同行评审中的懒惰思维和非具体批评问题，提出了一种基于大语言模型的框架，通过分解评审意见、识别违规并生成针对性反馈来提高评审质量。

Abstract

Peer review is central to scientific quality, yet reliance on simple heuristics -- lazy thinking -- has lowered standards. Prior work treats lazy thinking detection as a single-label task, but review segments may exhibit multiple issues, including broader clarity problems, or specificity issues. Turning detection into actionable improvements requires guideline-aware feedback, which is currently missing. We introduce an LLM-driven framework that decomposes reviews into argumentative segments, identifies issues via a neurosymbolic module combining LLM features with traditional classifiers, and generates targeted feedback using issue-specific templates refined by a genetic algorithm. Experiments show our method outperforms zero-shot LLM baselines and improves review quality by up to 92.4\%. We also release LazyReviewPlus, a dataset of 1,309 sentences labeled for lazy thinking and specificity.

Problem

• lazy thinking
• non-specific critiques
• review quality
• guideline compliance
• peer review

Innovation

• LLM-driven framework
• issue-specific templates
• iterative reranking-based generation algorithm
• LazyReviewPlus