Vibe Check: Understanding the Effects of LLM-Based Conversational Agents' Personality and Alignment on User Perceptions in Goal-Oriented Tasks

2025-09-11

AI Summary

This study investigates how the intensity of personality expression in large language model (LLM)-driven conversational agents (CAs) and the degree of user–agent personality alignment affect user perceptions in goal-oriented tasks. We propose the “Trait Modulation Key” framework to precisely calibrate CA personality expression across the Big Five dimensions and conduct a between-subjects experiment in a travel planning scenario, integrating multidimensional perception scales and cluster analysis. Results reveal an inverted-U relationship between personality expression alignment and user perceptions: moderate alignment significantly enhances perceived intelligence, enjoyment, anthropomorphism, trust, adoption intention, and likability; extraversion and emotional stability emerge as key moderating traits. Cluster analysis of user responses identifies three distinct compatibility archetypes, uncovering heterogeneous mechanisms of personality adaptation. These findings provide both theoretical grounding and practical design principles for interpretable, controllable, and personality-adaptive conversational agents.

Technology Category

• Humans and AI: Intelligent User Interfaces
• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Multiagent Systems: Agent Communication

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Agentic search

Abstract

Large language models (LLMs) enable conversational agents (CAs) to express distinctive personalities, raising new questions about how such designs shape user perceptions. This study investigates how personality expression levels and user-agent personality alignment influence perceptions in goal-oriented tasks. In a between-subjects experiment (N=150), participants completed travel planning with CAs exhibiting low, medium, or high expression across the Big Five traits, controlled via our novel Trait Modulation Keys framework. Results revealed an inverted-U relationship: medium expression produced the most positive evaluations across Intelligence, Enjoyment, Anthropomorphism, Intention to Adopt, Trust, and Likeability, significantly outperforming both extremes. Personality alignment further enhanced outcomes, with Extraversion and Emotional Stability emerging as the most influential traits. Cluster analysis identified three distinct compatibility profiles, with "Well-Aligned" users reporting substantially positive perceptions. These findings demonstrate that personality expression and strategic trait alignment constitute optimal design targets for CA personality, offering design implications as LLM-based CAs become increasingly prevalent.

Problem

• Investigating how LLM-based conversational agents' personality expression levels affect user perceptions
• Examining the impact of user-agent personality alignment on evaluations in goal-oriented tasks
• Identifying optimal personality design strategies for conversational agents using Big Five traits

Innovation

• Trait Modulation Keys framework
• Personality alignment enhances outcomes
• Inverted-U relationship expression levels