MedREK: Retrieval-Based Editing for Medical LLMs with Key-Aware Prompts

2025-10-15

AI Summary

Medical large language models (LLMs) face two critical challenges: knowledge obsolescence and hallucination—stemming from outdated or erroneous training data—and limitations of existing retrieval-based editing methods, including medical knowledge representation overlap and reliance on single-sample editing paradigms. To address these, we propose MedREK, the first retrieval-based editing framework for medical LLMs supporting both single-shot and batch editing. Its key contributions are: (1) a key-aware prompting mechanism to enhance retrieval precision and edit localization; (2) a shared query-key module coupled with an attention-driven prompt encoder for efficient batch knowledge correction; and (3) vector representation disentanglement to mitigate semantic overlap. Evaluated on MedVersa—a novel, multi-granularity medical editing benchmark—we demonstrate that MedREK significantly outperforms state-of-the-art methods, validating its effectiveness, strong locality, and clinical reliability in batch editing. Code and data are publicly available.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

LLMs hold great promise for healthcare applications, but the rapid evolution of medical knowledge and errors in training data often cause them to generate outdated or inaccurate information, limiting their applicability in high-stakes clinical practice. Model editing has emerged as a potential remedy without full retraining. While parameter-based editing often compromises locality and is thus ill-suited for the medical domain, retrieval-based editing offers a more viable alternative. However, it still faces two critical challenges: (1) representation overlap within the medical knowledge space often causes inaccurate retrieval and reduces editing accuracy; (2) existing methods are restricted to single-sample edits, while batch-editing remains largely unexplored despite its importance for real-world medical applications. To address these challenges, we first construct MedVersa, hk{an enhanced benchmark with broader coverage of medical subjects, designed to evaluate both single and batch edits under strict locality constraints}. We then propose MedREK, a retrieval-based editing framework that integrates a shared query-key module for precise matching with an attention-based prompt encoder for informative guidance. Experimental results on various medical benchmarks demonstrate that our MedREK achieves superior performance across different core metrics and provides the first validated solution for batch-editing in medical LLMs. Our code and dataset are available at https://github.com/mylittleriver/MedREK.

Problem

• Addresses outdated medical knowledge in LLMs
• Solves inaccurate retrieval in medical editing
• Enables batch-editing for clinical applications

Innovation

• Retrieval-based editing framework for medical LLMs
• Shared query-key module enables precise knowledge matching
• Attention-based prompt encoder provides informative guidance