Articulate Intuition or Genuine Analysis? Benchmarking Epistemic Reliability in LLM-as-a-Judge Peer Reviews

2026-07-11

AI Summary

This study addresses the critical distinction between superficial fluency and genuine analytical capability in large language model (LLM)-generated peer reviews, aiming to assess their credibility. Drawing on Kahneman’s dual-process theory, the authors operationalize this distinction into a structured review scoring framework and introduce Kahneman4Review—a benchmark dataset comprising 3,563 reviews annotated across nine textual dimensions, eight bias diagnostics, and continuous reasoning quality scores. Through structured rating scales, textual probe matching, bias metrics, and reasoning assessments, the analysis reveals that high-scoring LLM reviews are primarily driven by length and publication venue. Notably, ICLR reviews from 2022–2023 exhibit significant shifts in textual characteristics. The proposed framework effectively disentangles surface-level fluency from cognitive reliability, establishing a novel paradigm for evaluating LLM-generated review quality.

Abstract

When an LLM judge calls a peer review analytical and a human committee calls another review high quality, are they tracking the same thing? We argue they are not, and that the difference matters philosophically. We operationalise Kahneman's dual-process theory into a structured rubric for peer review and release Kahneman4Review, a benchmark of 3,563 rated reviews scored along nine theoretically motivated textual dimensions, eight bias diagnostics, and a continuous reasoning-quality score. Three findings bear on trustworthiness: decision tier is not detectably aligned with the rubric's text-grounded epistemic-quality proxy; public-showcase agentic reviews receive higher raw scores than pooled human reviews, but length and venue explain most of the gap and the samples are not paper-paired; and ICLR review-text diagnostics shift at the 2022--2023 transition, temporally coincident with widespread LLM availability but without identifying its cause. A matched function-probe pilot further shows that the rubric distinguishes textual probes designed to contrast genuine fault-finding with surface fluency. We argue that a trustworthy reliability benchmark for LLM judges must separate analytical form from epistemic function, and propose concrete design choices toward that goal. An interactive demo is available at https://huggingface.co/spaces/nuojohnchen/Kahneman4Review.

Problem

• LLM-as-a-Judge
• peer review
• epistemic reliability
• analytical form
• reasoning quality

Innovation

• LLM-as-a-Judge
• epistemic reliability
• dual-process theory
• peer review benchmark
• reasoning-quality evaluation