Hallucination Detection-Guided Preference Optimization for Clinical Summarization

2026-05-27

AI Summary

This work addresses the susceptibility of large language models to hallucinations in clinical summarization, which undermines their reliability in medical settings. The authors propose IterModel, a novel inference-time iterative refinement framework that leverages a hallucination detector to guide the generation of more factually accurate summaries. Crucially, the correction trajectories produced during this process are automatically converted into preference pairs for fine-tuning (referred to as Model), enabling factual consistency enhancement without manual annotation. This approach represents the first integration of hallucination detection signals into both iterative refinement and preference learning pipelines. Experimental results on MIMIC-IV clinical notes demonstrate that IterModel and its fine-tuned variant Model reduce the hallucination rate of Llama-3.1-8B-Instruct by 24% and 48%, respectively, while preserving summary fluency, coherence, and relevance.

Abstract

Large language models (LLMs) have shown promise on summarization tasks, but they often produce hallucinations, which are unsupported or incorrect statements that limit their reliability in specialized healthcare applications. We introduce \itermodelfull (\itermodel), an inference-time method that leverages hallucination detectors to guide iterative summary revisions toward factual corrections. Building on this, we propose \itermodel for Preference Learning (\model), which converts detector-guided refinement trajectories into preference pairs for model finetuning. Extensive experiments show that our methods substantially reduce hallucinations for Llama and Gemma models in summarizing real-world clinical notes from \MimicIV. For example, \itermodel reduces 24\% and \model reduces 48\% hallucinations in Llama-3.1-8B-Instruct. Importantly, both methods preserve summary fluency, coherence, and relevance according to human expert and LLM-Jury evaluations. Together, these results demonstrate that detection-informed refinement and preference learning offer an automated solution for improving factual faithfulness in clinical summarization.

Problem

• hallucination
• clinical summarization
• factual faithfulness
• large language models
• healthcare applications

Innovation

• hallucination detection
• preference optimization
• clinical summarization
• iterative refinement
• factual faithfulness