Empirical evidence of Large Language Model's influence on human spoken communication

2024-09-03

arXiv.org

AI Summary

This study investigates whether large language models (LLMs), such as ChatGPT, reshape human spoken language and cultural practices through human–AI linguistic feedback loops. Method: Leveraging ASR-transcribed speech from 280,000 university-level YouTube lecture videos, we conduct time-series word frequency analysis and employ a pre–post ChatGPT release quasi-experimental design. Contribution/Results: We present the first empirical evidence that LLMs directly influence authentic human spoken behavior: post-release, ChatGPT-characteristic lexical items exhibit statistically significant increases in academic speech (p < 0.001), confirming systematic oral imitation by humans. Moving beyond prior written-language–focused work, this study reveals the mechanism of AI-generated language diffusion into spoken discourse. It further highlights critical sociocultural risks—including erosion of linguistic diversity, discursive manipulation, and asymmetric human–AI co-evolution—thereby advancing foundational understanding of LLMs’ real-world linguistic impact.

Technology Category

• Natural Language Processing: (Large) Language Models
• Machine Learning: Large Multimodal Models (LMMs)
• Humans and AI: Learning Human Values and Preferences

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Artificial Intelligence (AI) agents now interact with billions of humans in natural language, thanks to advances in Large Language Models (LLMs) like ChatGPT. This raises the question of whether AI has the potential to shape a fundamental aspect of human culture: the way we speak. Recent analyses revealed that scientific publications already exhibit evidence of AI-specific language. But this evidence is inconclusive, since scientists may simply be using AI to copy-edit their writing. To explore whether AI has influenced human spoken communication, we transcribed and analyzed about 280,000 English-language videos of presentations, talks, and speeches from more than 20,000 YouTube channels of academic institutions. We find a significant shift in the trend of word usage specific to words distinctively associated with ChatGPT following its release. These findings provide the first empirical evidence that humans increasingly imitate LLMs in their spoken language. Our results raise societal and policy-relevant concerns about the potential of AI to unintentionally reduce linguistic diversity, or to be deliberately misused for mass manipulation. They also highlight the need for further investigation into the feedback loops between machine behavior and human culture.

Problem

• Investigates ChatGPT's impact on human spoken language patterns
• Measures lexicographic shifts in academic and podcast discourse post-ChatGPT
• Assesses risks of AI-driven cultural homogenization and manipulation

Innovation

• Econometric causal inference techniques applied
• Detects lexicographic shift in human speech
• Measures ChatGPT's cultural influence quantitatively