Who are you, ChatGPT? Personality and Demographic Style in LLM-Generated Content

2025-10-13

AI Summary

This study investigates whether generative large language models (LLMs) systematically exhibit human-like personality traits (e.g., agreeableness, neuroticism) and gendered linguistic patterns. Method: We propose a self-report-free, data-driven evaluation framework that constructs a large-scale human–model response contrast dataset using open-ended Reddit Q&A data, and applies automated personality recognition and gender classification to quantitatively analyze six mainstream LLMs. Contribution/Results: Our analysis reveals— for the first time—that LLMs consistently display a stable personality profile characterized by high agreeableness and low neuroticism. While their gendered language aligns broadly with human distributions, inter-gender differences are markedly attenuated. Overall, LLMs adopt a more cooperative, less confrontational conversational style. This framework establishes a novel paradigm for AI personality modeling and interpretable, behavior-based model assessment.

Technology Category

• Natural Language Processing: (Large) Language Models
• Machine Learning: Large Multimodal Models (LMMs)
• Humans and AI: Learning Human Values and Preferences

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems

Abstract

Generative large language models (LLMs) have become central to everyday life, producing human-like text across diverse domains. A growing body of research investigates whether these models also exhibit personality- and demographic-like characteristics in their language. In this work, we introduce a novel, data-driven methodology for assessing LLM personality without relying on self-report questionnaires, applying instead automatic personality and gender classifiers to model replies on open-ended questions collected from Reddit. Comparing six widely used models to human-authored responses, we find that LLMs systematically express higher Agreeableness and lower Neuroticism, reflecting cooperative and stable conversational tendencies. Gendered language patterns in model text broadly resemble those of human writers, though with reduced variation, echoing prior findings on automated agents. We contribute a new dataset of human and model responses, along with large-scale comparative analyses, shedding new light on the topic of personality and demographic patterns of generative AI.

Problem

• Assessing personality traits in LLMs without using self-report questionnaires
• Analyzing gendered language patterns in AI-generated versus human text
• Developing data-driven methodology to evaluate demographic characteristics in LLMs

Innovation

• Using automatic classifiers for personality assessment
• Applying Reddit data to analyze LLM responses
• Comparing six models with human-authored text