AutoPK: Leveraging LLMs and a Hybrid Similarity Metric for Advanced Retrieval of Pharmacokinetic Data from Complex Tables and Documents

2025-09-26

AI Summary

Pharmacokinetic (PK) tables exhibit complex structures and terminological heterogeneity, severely impeding automated data extraction and standardization. To address this, we propose AutoPK: a two-stage framework wherein Stage I leverages large language models (LLMs) to identify variant expressions of PK parameters, and Stage II integrates semantic similarity measurement, LLM-based validation, and key-value text transformation to achieve precise parameter normalization. Our key innovations include a hybrid similarity metric and a lightweight verification feedback loop, which substantially mitigate LLM hallucination. Evaluated on 605 real-world PK tables, AutoPK achieves F1-scores of 0.92 for half-life and 0.91 for clearance using LLaMA-3.1-70B. With the smaller Gemma-3-27B, it improves F1 by 2–7× and reduces hallucination rates from 60–95% to 8–14%, outperforming leading commercial systems.

Technology Category

• Machine Learning: Auto ML and Hyperparameter Tuning
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Pharmacokinetics (PK) plays a critical role in drug development and regulatory decision-making for human and veterinary medicine, directly affecting public health through drug safety and efficacy assessments. However, PK data are often embedded in complex, heterogeneous tables with variable structures and inconsistent terminologies, posing significant challenges for automated PK data retrieval and standardization. AutoPK, a novel two-stage framework for accurate and scalable extraction of PK data from complex scientific tables. In the first stage, AutoPK identifies and extracts PK parameter variants using large language models (LLMs), a hybrid similarity metric, and LLM-based validation. The second stage filters relevant rows, converts the table into a key-value text format, and uses an LLM to reconstruct a standardized table. Evaluated on a real-world dataset of 605 PK tables, including captions and footnotes, AutoPK shows significant improvements in precision and recall over direct LLM baselines. For instance, AutoPK with LLaMA 3.1-70B achieved an F1-score of 0.92 on half-life and 0.91 on clearance parameters, outperforming direct use of LLaMA 3.1-70B by margins of 0.10 and 0.21, respectively. Smaller models such as Gemma 3-27B and Phi 3-12B with AutoPK achieved 2-7 fold F1 gains over their direct use, with Gemma's hallucination rates reduced from 60-95% down to 8-14%. Notably, AutoPK enabled open-source models like Gemma 3-27B to outperform commercial systems such as GPT-4o Mini on several PK parameters. AutoPK enables scalable and high-confidence PK data extraction, making it well-suited for critical applications in veterinary pharmacology, drug safety monitoring, and public health decision-making, while addressing heterogeneous table structures and terminology and demonstrating generalizability across key PK parameters. Code and data: https://github.com/hosseinsholehrasa/AutoPK

Problem

• Extracting pharmacokinetic data from complex heterogeneous tables with variable structures
• Addressing inconsistent terminologies in automated PK data retrieval and standardization
• Improving precision and recall for PK parameter extraction from scientific documents

Innovation

• Uses LLMs and hybrid similarity for parameter extraction
• Converts tables to key-value format for standardization
• Validates and reconstructs tables with LLM-based filtering