Customizing Open Source LLMs for Quantitative Medication Attribute Extraction across Heterogeneous EHR Systems

2025-10-23

AI Summary

Monitoring Medication for Opioid Use Disorder (MOUD) prescriptions across heterogeneous electronic health records (EHRs) is hindered by the fragmentation of prescription attributes across structured fields and unstructured clinical notes, impeding cross-institutional surveillance. Method: We propose a lightweight, LLM-based structured information extraction framework leveraging open-source models—including Llama, Qwen, Gemma, and MedGemma—integrated with fixed JSON schema parsing, rule-based dose inference, dosing interval modeling, unit standardization, and novel normalization and cross-field consistency validation mechanisms, enabling privacy-preserving, on-premise deployment. Results: Evaluated on 25,605 real-world EHR records from five clinics, Qwen2.5-32B achieves 93.4% coverage and 93.0% exact-match accuracy; MedGemma-27B attains 93.1%/92.2%. Both significantly outperform legacy site-specific ETL pipelines, enhancing consistency, scalability, and robustness in multi-center MOUD exposure analysis.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Information Extraction
• Data Mining & Knowledge Management: Linked Open Data, Knowledge Graphs & KB Completion

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Harmonizing medication data across Electronic Health Record (EHR) systems is a persistent barrier to monitoring medications for opioid use disorder (MOUD). In heterogeneous EHR systems, key prescription attributes are scattered across differently formatted fields and freetext notes. We present a practical framework that customizes open source large language models (LLMs), including Llama, Qwen, Gemma, and MedGemma, to extract a unified set of MOUD prescription attributes (prescription date, drug name, duration, total quantity, daily quantity, and refills) from heterogeneous, site specific data and compute a standardized metric of medication coverage, emph{MOUD days}, per patient. Our pipeline processes records directly in a fixed JSON schema, followed by lightweight normalization and cross-field consistency checks. We evaluate the system on prescription level EHR data from five clinics in a national OUD study (25{,}605 records from 1{,}257 patients), using a previously annotated benchmark of 10{,}369 records (776 patients) as the ground truth. Performance is reported as coverage (share of records with a valid, matchable output) and record-level exact-match accuracy. Larger models perform best overall: Qwen2.5-32B achieves extbf{93.4%} coverage with extbf{93.0%} exact-match accuracy across clinics, and MedGemma-27B attains extbf{93.1%}/ extbf{92.2%}. A brief error review highlights three common issues and fixes: imputing missing dosage fields using within-drug norms, handling monthly/weekly injectables (e.g., Vivitrol) by setting duration from the documented schedule, and adding unit checks to prevent mass units (e.g., ``250 g'') from being misread as daily counts. By removing brittle, site-specific ETL and supporting local, privacy-preserving deployment, this approach enables consistent cross-site analyses of MOUD exposure, adherence, and retention in real-world settings.

Problem

• Extracting medication attributes from heterogeneous EHR systems
• Harmonizing opioid use disorder prescription data across clinics
• Customizing open source LLMs for standardized medication coverage metrics

Innovation

• Customizing open source LLMs for medication attribute extraction
• Processing records directly in fixed JSON schema format
• Applying lightweight normalization and cross-field consistency checks