Human--LLM Collaboration Is Transforming Complexity Metrics in Scientific Texts

2026-06-25

AI Summary

This study investigates how human–LLM collaboration reshapes the complexity characteristics of scientific texts following the widespread integration of large language models (LLMs). Drawing on a complex systems framework, we analyze millions of arXiv abstracts from 2010 to 2025, using 2023 as a breakpoint for comparative analysis. Employing Zipf’s law, Heaps’ law, lexical diversity metrics, and a newly developed LLM Style Index, we uncover for the first time that the rate of lexical turnover in scientific writing has significantly increased since 2023. Moreover, the previously strong positive correlations between the LLM Style Index and traditional complexity measures—such as vocabulary size, Zipf exponents, and Heaps exponents—have markedly weakened, indicating that human–LLM co-authorship is driving a structural evolution in scientific language.

Abstract

While human language has long been studied as a complex system, Large Language Models (LLMs) are rapidly becoming contributors to its dynamics. Because LLMs are trained on human language use, their effects on the broader human-AI linguistic ecosystem are likely subtle at first. As their use becomes more widespread, however, LLMs may alter emergent properties of language, particularly as models increasingly train on mixed human-LLM textual data. Here, we draw on complexity science to look for subtle LLM effects in millions of arXiv abstracts from 2010 to 2025. The year 2023, when LLMs rapidly became widely used, serves as a landmark in a natural experiment. While we find a sharp increase in a composite LLM-associated style index after early 2023, we observe only subtle changes in the exponents of Zipf's law and Heaps' law. More compelling, however, are two subtle changes in complexity metrics that emerge from 2023 onward. First, turnover among top-ranked words increases sharply. Second, the positive relationship between the LLM-associated style index and three complexity metrics--vocabulary size and the exponents of Heaps' and Zipf's laws--becomes flatter after 2022. Together, these patterns are consistent with changes in the emergent properties of scientific text in a mixed human-AI linguistic ecosystem.

Problem

• human-LLM collaboration
• complexity metrics
• scientific texts
• emergent properties
• linguistic ecosystem

Innovation

• human-LLM collaboration
• complexity metrics
• Zipf's law
• Heaps' law
• linguistic ecosystem