RAG-based Architectures for Drug Side Effect Retrieval in LLMs

2025-07-18

AI Summary

To address critical limitations of large language models (LLMs)—including knowledge gaps, frequent hallucinations, and poor interpretability—in drug adverse effect retrieval, this paper proposes GraphRAG, a novel framework integrating graph-structured knowledge with retrieval-augmented generation (RAG). GraphRAG leverages a prior drug–adverse-effect association graph to explicitly guide the Llama-3-8B model in modeling complex semantic relationships during retrieval. Evaluated on a benchmark comprising 19,520 real-world drug–adverse-effect pairs, GraphRAG achieves a 99.7% retrieval accuracy, substantially outperforming standard RAG and fine-tuned baselines. Moreover, it demonstrates strong scalability and decision traceability—enabling transparent, step-by-step justification of retrieved associations. By unifying structured biomedical knowledge with LLM-based reasoning, GraphRAG establishes a high-precision, interpretable paradigm for pharmacovigilance and pharmacoepidemiological analysis.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community
• Reasoning under Uncertainty: Graphical Models

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Drug side effects are a major global health concern, necessitating advanced methods for their accurate detection and analysis. While Large Language Models (LLMs) offer promising conversational interfaces, their inherent limitations, including reliance on black-box training data, susceptibility to hallucinations, and lack of domain-specific knowledge, hinder their reliability in specialized fields like pharmacovigilance. To address this gap, we propose two architectures: Retrieval-Augmented Generation (RAG) and GraphRAG, which integrate comprehensive drug side effect knowledge into a Llama 3 8B language model. Through extensive evaluations on 19,520 drug side effect associations (covering 976 drugs and 3,851 side effect terms), our results demonstrate that GraphRAG achieves near-perfect accuracy in drug side effect retrieval. This framework offers a highly accurate and scalable solution, signifying a significant advancement in leveraging LLMs for critical pharmacovigilance applications.

Problem

• Improving drug side effect retrieval accuracy in LLMs
• Addressing LLM limitations for pharmacovigilance applications
• Enhancing domain-specific knowledge integration in language models

Innovation

• RAG integrates drug side effect knowledge
• GraphRAG achieves near-perfect retrieval accuracy
• Scalable solution for pharmacovigilance applications