Adaptive Knowledge Graphs Enhance Medical Question Answering: Bridging the Gap Between LLMs and Evolving Medical Knowledge

2025-02-18

AI Summary

To address the low timeliness and reliability of medical knowledge in question answering—caused by outdated knowledge bases and slow manual updates—this paper proposes an automatically evolving framework for constructing and updating medical knowledge graphs. Methodologically, it integrates adaptive graph generation, retrieval-augmented generation (RAG) over heterogeneous biomedical literature sources (e.g., PubMed, WikiSearch), graph neural network–driven dynamic reasoning, and an LLM-coordinated retrieve-then-generate paradigm to enable real-time evidence integration and interpretable, evolution-aware inference. Its key contribution is the first continuous self-updating mechanism for knowledge graphs, significantly improving trustworthiness and temporal relevance without increasing computational overhead. Experiments demonstrate state-of-the-art performance: 74.1% F1 on MEDQA and 66.34% accuracy on MEDMCQA—surpassing both same-scale and models 10–100× larger—validating its efficiency, scalability, and methodological advancement.

Technology Category

• Data Mining & Knowledge Management: Linked Open Data, Knowledge Graphs & KB Completion
• Machine Learning: Graph-based Machine Learning
• Reasoning under Uncertainty: Graphical Models

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Large Language Models (LLMs) have significantly advanced medical question-answering by leveraging extensive clinical data and medical literature. However, the rapid evolution of medical knowledge and the labor-intensive process of manually updating domain-specific resources pose challenges to the reliability of these systems. To address this, we introduce Adaptive Medical Graph-RAG (AMG-RAG), a comprehensive framework that automates the construction and continuous updating of medical knowledge graphs, integrates reasoning, and retrieves current external evidence, such as PubMed and WikiSearch. By dynamically linking new findings and complex medical concepts, AMG-RAG not only improves accuracy but also enhances interpretability in medical queries. Evaluations on the MEDQA and MEDMCQA benchmarks demonstrate the effectiveness of AMG-RAG, achieving an F1 score of 74.1 percent on MEDQA and an accuracy of 66.34 percent on MEDMCQA, outperforming both comparable models and those 10 to 100 times larger. Notably, these improvements are achieved without increasing computational overhead, highlighting the critical role of automated knowledge graph generation and external evidence retrieval in delivering up-to-date, trustworthy medical insights.

Problem

• Adaptive Knowledge Graphs enhance medical QA accuracy.
• Automates medical knowledge graph construction and updates.
• Improves reliability of Large Language Models in medicine.

Innovation

• Automates medical knowledge graph updates
• Integrates reasoning with external evidence
• Enhances accuracy and interpretability dynamically