Detecting AI-Generated Content in Academic Peer Reviews

2026-01-30

AI Summary

This study investigates the detection of AI-generated content in academic peer review and its potential implications for scholarly evaluation systems. By training a large language model–based detector on historical review data from ICLR and Nature Communications spanning 2022 to 2025, and employing longitudinal time-series analysis, the work provides the first systematic tracking and quantification of the emergence and growth trajectory of AI-generated reviews. The findings reveal that by 2025, approximately 20% of ICLR reviews and 12% of Nature Communications reviews are classified as AI-generated, with the latter exhibiting a particularly pronounced increase during Q3–Q4 of 2024. These results underscore the rapid diffusion of AI-generated content into academic peer review processes.

Technology Category

• Natural Language Processing: Generation
• Humans and AI: Game Design — Procedural Content Generation & Storytelling
• Machine Learning: Deep Generative Models & Autoencoders

Application Category

• Web Mining and Content Analysis: Web data quality in the era of algorithmically-generated content
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

The growing availability of large language models (LLMs) has raised questions about their role in academic peer review. This study examines the temporal emergence of AI-generated content in peer reviews by applying a detection model trained on historical reviews to later review cycles at International Conference on Learning Representations (ICLR) and Nature Communications (NC). We observe minimal detection of AI-generated content before 2022, followed by a substantial increase through 2025, with approximately 20% of ICLR reviews and 12% of Nature Communications reviews classified as AI-generated in 2025. The most pronounced growth of AI-generated reviews in NC occurs between the third and fourth quarter of 2024. Together, these findings provide suggestive evidence of a rapidly increasing presence of AI-assisted content in peer review and highlight the need for further study of its implications for scholarly evaluation.

Problem

• AI-generated content
• peer review
• academic integrity
• large language models
• content detection

Innovation

• AI-generated content detection
• academic peer review
• large language models
• temporal analysis
• scholarly evaluation