Creating Multilingual Mental Health Dialogue Datasets: Limits of Persona-Based Localization via Nationality and Language

2026-06-17

AI Summary

The scarcity of high-quality multilingual mental health dialogue data hinders the equitable global development of digital psychological support systems. This study addresses this gap by leveraging large language models (LLMs) to generate synthetic dialogues in Chinese, Bengali, and Hindi, systematically controlling for clinician nationality and language parameters. Using LLMs as evaluators, the research assesses the models’ ability to judge depression severity across these languages. Findings reveal that merely adjusting nationality and language parameters fails to ensure cross-cultural clinical consistency, with significantly reduced depression assessment accuracy in non-English contexts and substantial performance variation across models. These results expose the systemic limitations of current English-centric approaches in multilingual settings and underscore the urgent need for culturally responsive, multilingual datasets to support fair and effective global mental health technologies.

Abstract

AI and large language models (LLMs) have emerged as promising tools to address global mental health challenges. Despite the global nature of these challenges, there remains a critical shortage of high-quality datasets for training and evaluating such systems. To mitigate this gap, researchers increasingly generate synthetic clinical personas to simulate user data and test digital mental health support systems. However, most validated personas rely on English-centric contexts. This paper investigates whether similar persona-based methods can be used to generate multilingual mental health datasets. We modified nationality and language parameters in personas to generate clinical dialogues in Mandarin, Bengali, and Hindi. We then examined how different LLMs perform when evaluating the depression severity of these generated multilingual datasets against the baseline in English. Our findings indicate that just adding nationality and language parameters in personas might not be adequate, as it can introduce clinical inconsistency across languages. LLM judge models often exhibit inaccuracies in assessing depression severity in non-English texts, with performance varying across different models. This exposes the systemic limitations of applying English-centric personas to multilingual contexts. Ultimately, our work highlights the urgent need for culturally responsive data generation to ensure equitable mental health systems globally.

Problem

• multilingual mental health
• persona-based localization
• clinical inconsistency
• LLM evaluation
• cultural responsiveness

Innovation

• multilingual mental health
• persona-based data generation
• large language models
• cultural localization
• clinical inconsistency