Multilingual KokoroChat: A Multi-LLM Ensemble Translation Method for Creating a Multilingual Counseling Dialogue Dataset

2026-03-24

AI Summary

High-quality, publicly available multilingual counseling dialogue data remain scarce, and individual large language models (LLMs) struggle to achieve high-fidelity translation in such high-sensitivity domains. To address this challenge, this work proposes a novel ensemble translation framework that first leverages multiple LLMs to generate diverse initial translations and then employs an additional LLM to evaluate and synthesize these candidates into a final, high-quality output. Experiments on the Japanese counseling corpus KokoroChat demonstrate that the proposed method significantly outperforms any single state-of-the-art LLM. Human preference evaluations further confirm its superior translation quality. The resulting Multilingual KokoroChat dataset has been publicly released, effectively filling a critical gap in multilingual resources for mental health counseling research.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Machine Translation, Multilinguality, Cross-Lingual NLP
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

To address the critical scarcity of high-quality, publicly available counseling dialogue datasets, we created Multilingual KokoroChat by translating KokoroChat, a large-scale manually authored Japanese counseling corpus, into both English and Chinese. A key challenge in this process is that the optimal model for translation varies by input, making it impossible for any single model to consistently guarantee the highest quality. In a sensitive domain like counseling, where the highest possible translation fidelity is essential, relying on a single LLM is therefore insufficient. To overcome this challenge, we developed and employed a novel multi-LLM ensemble method. Our approach first generates diverse hypotheses from multiple distinct LLMs. A single LLM then produces a high-quality translation based on an analysis of the respective strengths and weaknesses of all presented hypotheses. The quality of ``Multilingual KokoroChat'' was rigorously validated through human preference studies. These evaluations confirmed that the translations produced by our ensemble method were preferred from any individual state-of-the-art LLM. This strong preference confirms the superior quality of our method's outputs. The Multilingual KokoroChat is available at https://github.com/UEC-InabaLab/MultilingualKokoroChat.

Problem

• multilingual counseling dataset
• translation fidelity
• LLM ensemble
• dialogue corpus scarcity

Innovation

• multi-LLM ensemble
• translation fidelity
• counseling dialogue dataset
• multilingual NLP
• LLM collaboration