No Universal Courtesy: A Cross-Linguistic, Multi-Model Study of Politeness Effects on LLMs Using the PLUM Corpus

2026-04-17

AI Summary

This study investigates how large language models respond differently to user prompts varying in politeness levels and examines the cross-linguistic and cross-model generalizability of these effects. Grounded in Brown and Levinson’s politeness theory and Culpeper’s impoliteness framework, we evaluate five prominent models across English, Hindi, and Spanish, incorporating three types of dialogue history. Using PLUM—a newly constructed, open-source, multilingual, multi-tiered corpus manually validated for politeness—we empirically test six falsifiable hypotheses. Through a quantitative analysis of 22,500 prompt–response pairs across eight evaluative dimensions, we find that polite prompts improve response quality by approximately 11% on average, though this effect is strongly moderated by both language and model: English responses favor polite or direct phrasing, Hindi prefers deferential indirectness, and Spanish favors assertiveness; Llama exhibits the highest sensitivity to tone, whereas GPT demonstrates greater robustness.

Technology Category

• Natural Language Processing: (Large) Language Models
• Machine Learning: Large Multimodal Models (LMMs)
• Humans and AI: Intelligent User Interfaces

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems

Abstract

This paper explores the response of Large Language Models (LLMs) to user prompts with different degrees of politeness and impoliteness. The Politeness Theory by Brown and Levinson and the Impoliteness Framework by Culpeper form the basis of experiments conducted across three languages (English, Hindi, Spanish), five models (Gemini-Pro, GPT-4o Mini, Claude 3.7 Sonnet, DeepSeek-Chat, and Llama 3), and three interaction histories between users (raw, polite, and impolite). Our sample consists of 22,500 pairs of prompts and responses of various types, evaluated across five levels of politeness using an eight-factor assessment framework: coherence, clarity, depth, responsiveness, context retention, toxicity, conciseness, and readability. The findings show that model performance is highly influenced by tone, dialogue history, and language. While polite prompts enhance the average response quality by up to ~11% and impolite tones worsen it, these effects are neither consistent nor universal across languages and models. English is best served by courteous or direct tones, Hindi by deferential and indirect tones, and Spanish by assertive tones. Among the models, Llama is the most tone-sensitive (11.5% range), whereas GPT is more robust to adversarial tone. These results indicate that politeness is a quantifiable computational variable that affects LLM behaviour, though its impact is language- and model-dependent rather than universal. To support reproducibility and future work, we additionally release PLUM (Politeness Levels in Utterances, Multilingual), a publicly available corpus of 1,500 human-validated prompts across three languages and five politeness categories, and provide a formal supplementary analysis of six falsifiable hypotheses derived from politeness theory, empirically assessed against the dataset.

Problem

• politeness
• large language models
• cross-linguistic
• model robustness
• user prompts

Innovation

• politeness modeling
• cross-lingual evaluation
• large language models
• PLUM corpus
• human-AI interaction