🤖 AI Summary
In clinical research, manual extraction of structured clinical features from unstructured electronic health records (EHRs) is time-consuming, inefficient, and error-prone. To address this, we propose a privacy-preserving, modular, on-premises large language model (LLM) framework that integrates retrieval-augmented generation (RAG) with structured-output prompt engineering, enabling secure, scalable, containerized deployment in HIPAA-compliant environments. Our approach uniquely synergizes RAG with deterministic structured-response mechanisms for clinical text parsing—balancing domain adaptability and strict data privacy. Evaluated across multiple medical feature extraction tasks, the framework achieves high accuracy, substantially reducing manual annotation effort and improving data consistency. Notably, its systematic evaluation uncovered previously undetected systematic errors in prior human annotations, thereby validating both its reliability and its capacity for quality assurance and error discovery.
📝 Abstract
Manual chart review remains an extremely time-consuming and resource-intensive component of clinical research, requiring experts to extract often complex information from unstructured electronic health record (EHR) narratives. We present a secure, modular framework for automated structured feature extraction from clinical notes leveraging locally deployed large language models (LLMs) on institutionally approved, Health Insurance Portability and Accountability Act (HIPPA)-compliant compute infrastructure. This system integrates retrieval augmented generation (RAG) and structured response methods of LLMs into a widely deployable and scalable container to provide feature extraction for diverse clinical domains. In evaluation, the framework achieved high accuracy across multiple medical characteristics present in large bodies of patient notes when compared against an expert-annotated dataset and identified several annotation errors missed in manual review. This framework demonstrates the potential of LLM systems to reduce the burden of manual chart review through automated extraction and increase consistency in data capture, accelerating clinical research.