Configurable Clinical Information Extraction with Agentic RAG: What Works, What Breaks, and Why

2026-06-17

AI Summary

This study addresses the challenge of extracting clinically relevant information from heterogeneous electronic health records, where patient data are scattered across numerous unstructured documents and structured entries lacking document-level metadata. Conventional retrieval-augmented generation (RAG) approaches struggle to support temporal reasoning and cross-document dependency modeling under such conditions. To overcome these limitations, this work proposes ACIE, a locally deployed agent-based RAG system tailored for real-world clinical information extraction. ACIE explicitly handles missing metadata, enables cross-document reasoning, and provides traceable citations to ensure interpretability and verifiability. Evaluated on 7,326 clinical judgments, the system achieved an overall physician acceptance rate of 96.5%, with per-category acceptance rates ranging from 80% to 99%.

Abstract

Patient contexts span hundreds of heterogeneous documents and thousands of structured data points, yet the document-level metadata that AI systems need for retrieval and triage is absent or incomplete. Standard retrieval-augmented generation fails on this data, mishandling temporal reasoning, cross-document dependencies, and missing metadata. We deploy ACIE (Agentic Clinical Information Extraction) at University Medicine Essen: an on-premise agentic RAG pipeline that reasons over complete patient contexts and grounds every answer in source passages for clinician verification. We quantify the metadata gap, trace the architectural decisions it shaped, and evaluate extraction alongside an independent retrospective lymphoma registry study, in which nuclear-medicine physicians verify every extracted value against its cited sources. Across 7,326 judgments, clinicians accepted 96.5\% of extractions, with per-type acceptance ranging from 80\% to 99\%.

Problem

• Clinical Information Extraction
• Retrieval-Augmented Generation
• Metadata Gap
• Temporal Reasoning
• Cross-document Dependencies

Innovation

• Agentic RAG
• Clinical Information Extraction
• Metadata Gap
• Temporal Reasoning
• Source-Grounded Verification