"Where does it hurt?" - Dataset and Study on Physician Intent Trajectories in Doctor Patient Dialogues

2025-08-26

AI Summary

This study investigates the evolution of clinicians’ clinical-reasoning-based intentions during patient–physician dialogues. We propose the first fine-grained clinician intention taxonomy grounded in the SOAP (Subjective, Objective, Assessment, Plan) framework and release a high-quality dataset of 5,000+ dialogue turns, annotated via expert medical review and Prolific crowdsourcing. For the first time, we systematically characterize intention transition patterns across SOAP phases. Leveraging this resource, we establish a joint benchmark for intention recognition and dialogue summarization, evaluating both generative and encoder-based models. Results show that our dataset significantly improves structural consistency in medical dialogue summarization (+12.3 ROUGE-L), and models effectively capture global dialogue flow—yet remain challenged in precisely identifying SOAP phase boundaries. This work provides both a theoretical foundation and critical resources for developing interpretable, diagnosis-driven clinical dialogue systems.

Technology Category

• Natural Language Processing: Conversational AI/Dialog Systems
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding
• Web Mining and Content Analysis: Normalization, clustering, classification, and summarization of Web text

Abstract

In a doctor-patient dialogue, the primary objective of physicians is to diagnose patients and propose a treatment plan. Medical doctors guide these conversations through targeted questioning to efficiently gather the information required to provide the best possible outcomes for patients. To the best of our knowledge, this is the first work that studies physician intent trajectories in doctor-patient dialogues. We use the `Ambient Clinical Intelligence Benchmark' (Aci-bench) dataset for our study. We collaborate with medical professionals to develop a fine-grained taxonomy of physician intents based on the SOAP framework (Subjective, Objective, Assessment, and Plan). We then conduct a large-scale annotation effort to label over 5000 doctor-patient turns with the help of a large number of medical experts recruited using Prolific, a popular crowd-sourcing platform. This large labeled dataset is an important resource contribution that we use for benchmarking the state-of-the-art generative and encoder models for medical intent classification tasks. Our findings show that our models understand the general structure of medical dialogues with high accuracy, but often fail to identify transitions between SOAP categories. We also report for the first time common trajectories in medical dialogue structures that provide valuable insights for designing `differential diagnosis' systems. Finally, we extensively study the impact of intent filtering for medical dialogue summarization and observe a significant boost in performance. We make the codes and data, including annotation guidelines, publicly available at https://github.com/DATEXIS/medical-intent-classification.

Problem

• Studying physician intent trajectories in doctor-patient dialogues
• Developing a fine-grained taxonomy of physician intents using SOAP framework
• Benchmarking models for medical intent classification and dialogue summarization

Innovation

• Developed fine-grained physician intent taxonomy using SOAP framework
• Conducted large-scale annotation of 5000+ doctor-patient dialogue turns
• Benchmarked generative and encoder models for medical intent classification