Conversation as Measurement in Clinical Encounters: Observable Phase Structure, Partially Observable Patient State

2026-08-09

AI Summary

This study investigates whether patient symptom states and dialogue phase structures can be reliably observed from clinical conversation transcripts alone. Leveraging a corpus of 439 real-world clinical dialogues (134 hours) and using patient-reported outcome measures (PROMs) as external anchors, the work presents the first large-scale systematic validation of the observability of these two dimensions in clinical language data. Automated annotations generated by a PHI-compliant GPT-5 model—refined through 40 hours of manual verification—were combined with PROM-based quantification of patient states and dialogue phase segmentation techniques. Results demonstrate that dialogue phase structure is highly observable and effectively captures the clinical workflow, whereas internal symptom states (e.g., voice quality, coughing, swallowing) are only partially observable, revealing significant limitations in inferring patient conditions solely from textual dialogue and underscoring critical caveats for deploying clinical language models.

Abstract

Many modern AI systems analyze conversational traces to infer aspects of human interaction and state, implicitly assuming that such information is recoverable from conversation. We study observability: whether a target is recoverable from conversational transcripts alone. Observability is difficult to assess because transcripts may provide only a partial view of many targets, and large-scale analysis requires model-based annotation, making true limits of the conversational signal hard to distinguish from annotator error. We therefore study clinical encounters, where patient-reported outcome measures (PROMs) provide an external anchor for patient state, and visits follow broadly structured patterns. We study observability of patient state and conversational phase structure using 439 real-world clinical encounter transcripts spanning 134 hours, including 245 ENT transcripts paired with 273 PROM surveys. We operationalize patient state using PROM scores for voice, cough, and swallowing; phase structure using conversational phase segmentation. To make these analyses credible at scale, we use a PHI-compliant GPT-5 deployment for transcript annotation and conduct 40 hours of manual validation, reducing the risk that apparent limits of observability simply reflect annotator error. Our core finding is an observability asymmetry: phase structure is observable and useful for characterizing clinical encounter organization, while patient state is only partially observable, even in a setting designed to elicit patient symptoms and experiences, cautioning against transcript-only inference of human state.

Problem

• observability
• clinical conversation
• patient state
• conversational phase structure
• transcript analysis

Innovation

• observability
• clinical conversation analysis
• patient-reported outcome measures (PROMs)
• conversational phase segmentation
• PHI-compliant LLM annotation