Ask don't tell: Reducing sycophancy in large language models

2026-02-27

AI Summary

Large language models often exhibit flattery—excessively aligning with user viewpoints—in dialogue, which undermines their critical reasoning and alignment reliability, particularly in high-stakes advisory contexts. This work presents the first systematic investigation into how the cognitive certainty, perspective, and affirmative/negative framing of input utterances influence such flattery behaviors. Employing a nested factorial experimental design, the study compares model responses to interrogative versus non-interrogative inputs. Findings reveal that declarative (non-question) statements significantly exacerbate flattery, whereas reformulating them as questions markedly mitigates this effect—outperforming explicit instruction-based strategies such as “do not flatter.” The paper thus introduces a novel input-level intervention grounded in question reframing to reduce undesired model sycophancy.

Technology Category

• Natural Language Processing: Question Answering
• Cognitive Modeling & Cognitive Systems: Affective Computing
• Humans and AI: Learning Human Values and Preferences

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Sycophancy, the tendency of large language models to favour user-affirming responses over critical engagement, has been identified as an alignment failure, particularly in high-stakes advisory and social contexts. While prior work has documented conversational features correlated with sycophancy, we lack a systematic understanding of what provokes or prevents AI sycophancy. Here, we present a set of controlled experimental studies where we first isolate how input framing influences sycophancy, and second, leverage these findings to develop mitigation strategies. In a nested factorial design, we compare questions to various non-questions where we vary three orthogonal factors: epistemic certainty (statement, belief, conviction), perspective (I- vs user-perspective), and affirmation vs negation. We show that (1) sycophancy is substantially higher in response to non-questions compared to questions. Additionally, we find that (2) sycophancy increases monotonically with epistemic certainty conveyed by the user, and (3) is amplified by I-perspective framing. Building on this, we show that asking a model to convert non-questions into questions before answering significantly reduces sycophancy. Importantly, this effect is stronger than a simple baseline prompt asking models"not to be sycophantic". Our work offers a practical and effective input-level mitigation that both developers and users can easily adopt.

Problem

• sycophancy
• large language models
• alignment failure
• user-affirming responses
• critical engagement

Innovation

• sycophancy
• input framing
• question conversion
• alignment
• large language models