KokoroChat: A Japanese Psychological Counseling Dialogue Dataset Collected via Role-Playing by Trained Counselors

2025-06-02

AI Summary

Existing counseling dialogue datasets face three key challenges: low quality in crowdsourced data, privacy and ethical risks associated with real clinical data, and insufficient authenticity and diversity in LLM-generated data. To address these, we introduce KokoroChat—the first Japanese long-term counseling dialogue dataset (6,589 dialogues) constructed via professional counselor-led role-playing, each accompanied by structured client feedback. We propose a novel role-playing protocol grounded in clinical training frameworks, ensuring authenticity, cultural appropriateness, and ethical safety. Fine-tuning open-source LLMs on KokoroChat yields significant improvements over baselines in empathy expression, problem identification, and intervention suggestion; automated evaluation confirms across-the-board gains in response quality. The dataset is publicly released and has been widely adopted by the research community.

Technology Category

• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Natural Language Processing: Conversational AI/Dialog Systems
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Generating psychological counseling responses with language models relies heavily on high-quality datasets. Crowdsourced data collection methods require strict worker training, and data from real-world counseling environments may raise privacy and ethical concerns. While recent studies have explored using large language models (LLMs) to augment psychological counseling dialogue datasets, the resulting data often suffers from limited diversity and authenticity. To address these limitations, this study adopts a role-playing approach where trained counselors simulate counselor-client interactions, ensuring high-quality dialogues while mitigating privacy risks. Using this method, we construct KokoroChat, a Japanese psychological counseling dialogue dataset comprising 6,589 long-form dialogues, each accompanied by comprehensive client feedback. Experimental results demonstrate that fine-tuning open-source LLMs with KokoroChat improves both the quality of generated counseling responses and the automatic evaluation of counseling dialogues. The KokoroChat dataset is available at https://github.com/UEC-InabaLab/KokoroChat.

Problem

• Lack of high-quality psychological counseling dialogue datasets
• Privacy and ethical concerns in real-world counseling data collection
• Limited diversity and authenticity in LLM-generated counseling dialogues

Innovation

• Role-playing by trained counselors for data collection
• High-quality Japanese psychological counseling dialogue dataset
• Fine-tuning LLMs improves counseling response quality