🤖 AI Summary
The increasing human-likeness of large language model (LLM)-generated text poses growing challenges for reliable authorship attribution and AI content detection.
Method: We propose a multi-level textual characterization framework integrating morphological, syntactic, and semantic features with stylistic embeddings. Conducting systematic quantitative analysis across eight domains and eleven mainstream LLMs, we incorporate interpretable linguistic metrics—including dependency distance and sentiment polarity—and combine statistical modeling with controlled sampling strategies.
Contribution/Results: We empirically demonstrate that human-written texts exhibit simpler syntactic structures yet higher semantic richness and stylistic diversity, whereas LLM outputs grow increasingly homogeneous across model generations, with diminishing inter-model stylistic divergence. This work provides the first multi-granular, linguistically grounded empirical evidence of fundamental human–machine textual differences, offering interpretable theoretical foundations and actionable technical pathways for text provenance analysis and AI content governance.
📝 Abstract
The rapid advancements in large language models (LLMs) have significantly improved their ability to generate natural language, making texts generated by LLMs increasingly indistinguishable from human-written texts. While recent research has primarily focused on using LLMs to classify text as either human-written and machine-generated texts, our study focus on characterizing these texts using a set of linguistic features across different linguistic levels such as morphology, syntax, and semantics. We select a dataset of human-written and machine-generated texts spanning 8 domains and produced by 11 different LLMs. We calculate different linguistic features such as dependency length and emotionality and we use them for characterizing human-written and machine-generated texts along with different sampling strategies, repetition controls and model release date. Our statistical analysis reveals that human-written texts tend to exhibit simpler syntactic structures and more diverse semantic content. Furthermore, we calculate the variability of our set of features across models and domains. Both human and machine texts show stylistic diversity across domains, with humans displaying greater variation in our features. Finally, we apply style embeddings to further test variability among human-written and machine-generated texts. Notably, newer models output text that is similarly variable, pointing to an homogenization of machine-generated texts.