SIMAX: A Scalable and Interpretable Framework for Multi-Fidelity and Annotated Clinician-Patient Dialogue Simulation

2026-06-29

AI Summary

This work addresses the high annotation cost and poor inter-annotator consistency in clinical physician–patient dialogue datasets, which hinder the evaluation of AI-based communication coding systems. To overcome these limitations, the authors propose a controllable generation framework for simulating clinician–patient dialogues with embedded behavioral annotations. The framework leverages predefined clinical scenarios, role-specific characteristics, and target communication behaviors, guided by dual codebooks—Global and WISER—and integrates speech synthesis with automatic audio quality assessment metrics (UTMOS, WV-MOS, WER, CER) and CLAP-based text–audio alignment. This approach enables, for the first time, multi-fidelity, interpretable, and reproducible clinical dialogue simulation. The system generates 3,388 cross-specialty dialogues, which automatic and human evaluations confirm exhibit high naturalness, transcription accuracy, and clinical authenticity, while also exposing limited sensitivity of existing coding systems along certain behavioral dimensions.

Abstract

Background. The widespread deployment of ambient digital scribes is driving large-scale capture of clinician-patient dialogues. Human coding of clinical communication data remains costly, inconsistent, and difficult to scale, motivating AI-driven communication coding systems. However, evaluating these systems requires real-world dialogues and human-coded labels, both hard to obtain at scale. Methods. We developed SIMAX (Scalable and Interpretable Framework for Multi-Fidelity and Annotated Clinician-Patient Dialogue Simulation), a framework for generating controlled clinical dialogue data with reference behavioral annotations. SIMAX generates clinician-patient dialogues from predefined clinical scenarios, personas and voice conditions, and target communication behaviors. Behaviors are controlled using two codebooks: the Global Codebook for overall communication quality and the WISER Codebook for specific countable behaviors. We evaluated SIMAX using automated and human quality assessments and an example communication coding system. Results. SIMAX generated 3,388 simulated dialogues across three specialties, multiple visit stages, persona characteristics, and accent conditions. Automated assessment showed mean UTMOS and WV-MOS scores of 3.03 and 2.61, WER and CER of 0.07 and 0.05, and CLAP cosine similarity of 0.41, suggesting reasonable speech naturalness, high transcription fidelity, and positive text-audio correspondence. Human evaluation showed a median MOS of 4.67 and a median clinical realism score of 3.00. Downstream evaluation suggests that SIMAX can assess how a communication coding system responds to behavioral targets and reveal insufficient sensitivity in some dimensions. Conclusions. SIMAX generates controlled and reproducible simulated clinician-patient dialogues, providing a data foundation for developing, validating, and refining communication coding systems.

Problem

• clinical dialogue simulation
• communication coding
• multi-fidelity data
• behavioral annotation
• scalable evaluation

Innovation

• multi-fidelity simulation
• interpretable dialogue generation
• behavioral codebooks
• clinician-patient communication
• scalable synthetic data