TalkDep: Clinically Grounded LLM Personas for Conversation-Centric Depression Screening

2025-08-06

AI Summary

Depression screening faces dual challenges: scarcity of real-world training data and insufficient clinical validity of existing virtual patients. To address these, we propose a clinician-in-the-loop virtual patient modeling framework that integrates DSM-5 diagnostic criteria, the PHQ-9 scale, and multidimensional contextual factors, enabling conditional, personalized symptom response generation via large language models. Our approach balances clinical credibility with conversational naturalness, supporting fine-grained modeling of symptom severity gradients and phenotypic diversity. Blind evaluation by board-certified psychiatrists demonstrates that our virtual patients significantly outperform baselines in diagnostic consistency (Cohen’s κ = 0.82), symptom authenticity, and role richness. This work provides a scalable, empirically validated resource for training and evaluating AI-based depression screening systems.

Technology Category

• Humans and AI: Game Design — Virtual Humans, NPCs and Autonomous Characters
• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

The increasing demand for mental health services has outpaced the availability of real training data to develop clinical professionals, leading to limited support for the diagnosis of depression. This shortage has motivated the development of simulated or virtual patients to assist in training and evaluation, but existing approaches often fail to generate clinically valid, natural, and diverse symptom presentations. In this work, we embrace the recent advanced language models as the backbone and propose a novel clinician-in-the-loop patient simulation pipeline, TalkDep, with access to diversified patient profiles to develop simulated patients. By conditioning the model on psychiatric diagnostic criteria, symptom severity scales, and contextual factors, our goal is to create authentic patient responses that can better support diagnostic model training and evaluation. We verify the reliability of these simulated patients with thorough assessments conducted by clinical professionals. The availability of validated simulated patients offers a scalable and adaptable resource for improving the robustness and generalisability of automatic depression diagnosis systems.

Problem

• Address shortage of real training data for depression diagnosis
• Generate clinically valid and diverse virtual patient responses
• Improve robustness of automatic depression diagnosis systems

Innovation

• Clinician-in-the-loop patient simulation pipeline
• LLM conditioned on psychiatric diagnostic criteria
• Diversified patient profiles for authentic responses