The Persona Paradox: Medical Personas as Behavioral Priors in Clinical Language Models

2026-01-08

arXiv.org

AI Summary

This study investigates how medical role assignments (e.g., emergency physician, nurse) and interaction styles (assertive vs. cautious) influence the professionalism and safety of clinical large language models (LLMs) in high-stakes decision-making. Employing a multi-model comparison, a multidimensional evaluation framework (encompassing task accuracy, calibration, and risk behavior), and both automated LLM-based and human clinician assessments (Cohen’s κ = 0.43), the work reveals—for the first time—a context-dependent, non-monotonic effect of role prompting: it improves accuracy and calibration by up to 20% in critical triage scenarios but degrades performance by a comparable margin in primary care settings. These findings underscore the necessity of carefully aligning role priors with clinical context and highlight the current inadequacy of model reasoning quality to earn clinical trust, as 95.9% of evaluations expressed low confidence.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Safety and Robustness
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Persona conditioning can be viewed as a behavioral prior for large language models (LLMs) and is often assumed to confer expertise and improve safety in a monotonic manner. However, its effects on high-stakes clinical decision-making remain poorly characterized. We systematically evaluate persona-based control in clinical LLMs, examining how professional roles (e.g., Emergency Department physician, nurse) and interaction styles (bold vs.\ cautious) influence behavior across models and medical tasks. We assess performance on clinical triage and patient-safety tasks using multidimensional evaluations that capture task accuracy, calibration, and safety-relevant risk behavior. We find systematic, context-dependent, and non-monotonic effects: Medical personas improve performance in critical care tasks, yielding gains of up to $\sim+20\%$ in accuracy and calibration, but degrade performance in primary-care settings by comparable margins. Interaction style modulates risk propensity and sensitivity, but it's highly model-dependent. While aggregated LLM-judge rankings favor medical over non-medical personas in safety-critical cases, we found that human clinicians show moderate agreement on safety compliance (average Cohen's $\kappa = 0.43$) but indicate a low confidence in 95.9\% of their responses on reasoning quality. Our work shows that personas function as behavioral priors that introduce context-dependent trade-offs rather than guarantees of safety or expertise. The code is available at https://github.com/rsinghlab/Persona\_Paradox.

Problem

• clinical language models
• persona conditioning
• behavioral priors
• clinical decision-making
• safety trade-offs

Innovation

• persona conditioning
• clinical language models
• behavioral priors
• context-dependent effects
• safety trade-offs