Problem
Research questions and friction points this paper is trying to address.
Hallucinations
Scientific Peer Reviews
Large Language Models
Review Reliability
HalluPeer: A Taxonomy-driven Benchmark for Detecting Hallucinations in Scientific Peer Reviews
🤖 AI Summary
为解决学术评审中大语言模型产生无根据陈述的问题，本文提出HalluPeer基准，通过分类和定位方法检测这些虚假信息。
📝 Abstract
The growing scale of academic peer review has motivated the use of Large Language Models (LLMs) as review assistants, yet LLMs can generate fluent but unsupported claims that undermine review reliability. Existing hallucination benchmarks are not designed for peer review, where verification requires grounding claims in long, technical papers. We introduce HalluPeer, a benchmark for detecting hallucinations in scientific peer reviews, providing aligned triples of paper content, human-written reviews, and hallucination-injected reviews, annotated for detection, classification, and localization. Our pipeline induces a peer-review-specific hallucination taxonomy, identifies review contexts, and injects hallucinations with automated filtering. Experiments on 12K papers and 38K reviews show that existing detectors struggle to separate hallucinations from legitimate critique, while evaluation on authentic reviews demonstrates that HalluPeer-defined hallucination patterns occur in real peer reviews, highlighting the critical need for source-aware verification. Our project page can be found in https://github.com/Lin-TzuLing/HalluPeer.git
Innovation
Methods, ideas, or system contributions that make the work stand out.
HalluPeer
Taxonomy-driven Benchmark
Scientific Peer Reviews
Hallucination Detection
Automated Filtering
🔎 Similar Papers
No similar papers found.
