Assessing the Limitations of Large Language Models in Clinical Fact Decomposition

2024-12-17

arXiv.org

AI Summary

Clinical fact decomposition—breaking down complex clinical statements into verifiable atomic facts—is critical for ensuring the safety of LLM-based medical applications; however, existing work lacks systematic investigation tailored to terminology-rich, multi-source heterogeneous electronic health records (EHRs). This paper introduces FactEHR, the first full-document-level clinical fact decomposition benchmark, comprising 2,168 clinical notes from four note types across multiple hospitals. We conduct the first systematic evaluation of mainstream LLMs on this task, revealing up to a 2.6× disparity in the number of generated atomic facts across models—highlighting their unreliability. To ensure rigor, we design a fine-grained decomposition protocol integrating clinical expert collaboration and multi-dimensional quality assessment. Our key contributions include: (1) releasing the first open-source clinical fact decomposition dataset, (2) an accompanying evaluation framework and codebase, thereby filling a foundational gap in medical fact verification resources and advancing trustworthy LLM research in healthcare.

Technology Category

• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Machine Learning: Large Multimodal Models (LMMs)
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling

Abstract

Verifying factual claims is critical for using large language models (LLMs) in healthcare. Recent work has proposed fact decomposition, which uses LLMs to rewrite source text into concise sentences conveying a single piece of information, as an approach for fine-grained fact verification. Clinical documentation poses unique challenges for fact decomposition due to dense terminology and diverse note types. To explore these challenges, we present FactEHR, a dataset consisting of full document fact decompositions for 2,168 clinical notes spanning four types from three hospital systems. Our evaluation, including review by clinicians, highlights significant variability in the quality of fact decomposition for four commonly used LLMs, with some LLMs generating 2.6x more facts per sentence than others. The results underscore the need for better LLM capabilities to support factual verification in clinical text. To facilitate future research in this direction, we plan to release our code at url{https://github.com/som-shahlab/factehr}.

Problem

• Evaluating factuality in clinical notes using LLMs
• Decomposing complex clinical statements into atomic facts
• Addressing challenges in clinical documentation for fact verification

Innovation

• LLM-based fact decomposition for clinical notes
• FactEHR dataset with 987,266 entailment pairs
• Clinician-reviewed evaluation of LLM performance