Most biomedical publications show signs of LLM-assisted writing

2026-08-11

AI Summary

This study addresses the critical gap in effective methods for monitoring the use of large language models (LLMs) in academic writing, which hinders informed policy development. The authors propose an unbiased estimation framework based on temporal shifts in lexical frequency, integrating statistical language modeling with large-scale text mining to objectively quantify signs of LLM-assisted writing in open-access biomedical articles from PubMed Central. Their analysis reveals that, by the end of 2025, 89% of papers exhibit significant overrepresentation of LLM-associated vocabulary. Notably, the likelihood of LLM use in the discussion section (68%) is approximately twice that in the methods section (32%), though the latter still shows a penetration rate exceeding 50%. This approach overcomes the limitations of existing detection tools that rely on biased signals, offering a scalable new paradigm for academic integrity oversight.

Abstract

Over the past several years, LLM-powered chatbots and agents have become widely used as a tool for academic writing. LLM-assisted writing can be valuable by removing language barriers but at the same time causes concerns about misconduct and fraud. To inform policy decisions, it is necessary to monitor the prevalence of LLM-altered texts in scholarly publications. Despite some recent progress in this direction, no existing method can produce reliable estimates. Here we suggest and validate a new unbiased approach to estimate LLM usage in a corpus of texts based on changing word frequencies. We apply our method to the full texts of open-access biomedical papers from Pubmed Central, and show that by the end of 2025, 89% of papers show excess of LLM-associated vocabulary. We also find that LLMs are twice as likely to be used when writing a paragraph in the Discussion section (68%) compared to a paragraph in the Methods section (32%), but even inside the Methods section, the overall prevalence of LLM usage is over 50%. We believe that our estimates are crucial to shape future guidelines and policies.

Problem

• LLM-assisted writing
• academic integrity
• text prevalence estimation
• scholarly publications
• biomedical literature

Innovation

• LLM detection
• word frequency analysis
• unbiased estimation
• academic writing
• biomedical literature