Comparing human and LLM politeness strategies in free production

2025-06-11

AI Summary

Large language models (LLMs) exhibit pragmatic misalignment in polite language use—systematically over-relying on negative politeness strategies (e.g., hedging, indirectness) even in positive contexts where positive strategies (e.g., praise, empathy) are socially appropriate. Method: We investigate this phenomenon through constrained and open-ended generation tasks, integrating computational pragmatics analysis, cross-scale model comparisons (≥70B parameters), and rigorous human evaluation. Contribution/Results: This work is the first to empirically uncover this implicit bias, challenging the “bigger-is-more-human-like” assumption. Quantitative and qualitative analyses reveal that while 70B+ models receive higher human preference scores, their politeness strategy distributions significantly deviate from human benchmarks and lack contextual sensitivity—exposing a fundamental alignment deficit. We propose pragmatic misalignment as a critical dimension for evaluating LLM social competence, advancing fine-grained assessment frameworks and alignment optimization for socially situated language generation.

Technology Category

• Natural Language Processing: (Large) Language Models
• Machine Learning: Large Multimodal Models (LMMs)
• Humans and AI: Learning Human Values and Preferences

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Polite speech poses a fundamental alignment challenge for large language models (LLMs). Humans deploy a rich repertoire of linguistic strategies to balance informational and social goals -- from positive approaches that build rapport (compliments, expressions of interest) to negative strategies that minimize imposition (hedging, indirectness). We investigate whether LLMs employ a similarly context-sensitive repertoire by comparing human and LLM responses in both constrained and open-ended production tasks. We find that larger models ($ge$70B parameters) successfully replicate key preferences from the computational pragmatics literature, and human evaluators surprisingly prefer LLM-generated responses in open-ended contexts. However, further linguistic analyses reveal that models disproportionately rely on negative politeness strategies even in positive contexts, potentially leading to misinterpretations. While modern LLMs demonstrate an impressive handle on politeness strategies, these subtle differences raise important questions about pragmatic alignment in AI systems.

Problem

• Comparing human and LLM politeness strategies in communication
• Assessing LLM alignment with human pragmatic politeness norms
• Identifying LLM overuse of negative politeness in positive contexts

Innovation

• Compare human and LLM politeness strategies
• Analyze context-sensitive linguistic repertoires
• Reveal LLM overuse of negative politeness