META-RAG: Meta-Analysis-Inspired Evidence-Re-Ranking Method for Retrieval-Augmented Generation in Evidence-Based Medicine

2025-10-27

AI Summary

Existing retrieval-augmented generation (RAG) methods struggle to identify high-quality clinical evidence in evidence-based medicine (EBM), limiting diagnostic accuracy and reliability. To address this, we propose a meta-analysis-inspired evidence re-ranking framework that systematically integrates reliability analysis, heterogeneity assessment, and generalizability evaluation into the RAG pipeline—establishing the first multi-criteria evidence filtering mechanism tailored for EBM. Our approach performs multidimensional evidence evaluation and dynamic re-ranking over PubMed search results. Experiments across multiple clinical decision-making benchmarks demonstrate up to a 11.4% absolute improvement in model accuracy, significantly enhancing the credibility, robustness, and clinical applicability of generated outputs. The core innovation lies in formalizing fundamental EBM appraisal paradigms as computable RAG components, thereby bridging meta-analytic principles with large language model–based clinical reasoning.

Technology Category

• Search and Optimization: Metareasoning and Metaheuristics
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Web Mining and Content Analysis: Web data provenance, reliability, and authenticity

Abstract

Evidence-based medicine (EBM) holds a crucial role in clinical application. Given suitable medical articles, doctors effectively reduce the incidence of misdiagnoses. Researchers find it efficient to use large language models (LLMs) techniques like RAG for EBM tasks. However, the EBM maintains stringent requirements for evidence, and RAG applications in EBM struggle to efficiently distinguish high-quality evidence. Therefore, inspired by the meta-analysis used in EBM, we provide a new method to re-rank and filter the medical evidence. This method presents multiple principles to filter the best evidence for LLMs to diagnose. We employ a combination of several EBM methods to emulate the meta-analysis, which includes reliability analysis, heterogeneity analysis, and extrapolation analysis. These processes allow the users to retrieve the best medical evidence for the LLMs. Ultimately, we evaluate these high-quality articles and show an accuracy improvement of up to 11.4% in our experiments and results. Our method successfully enables RAG to extract higher-quality and more reliable evidence from the PubMed dataset. This work can reduce the infusion of incorrect knowledge into responses and help users receive more effective replies.

Problem

• Improves evidence quality filtering for medical RAG systems
• Enhances reliability of retrieved evidence using meta-analysis principles
• Reduces incorrect knowledge infusion in evidence-based medicine responses

Innovation

• Meta-analysis-inspired evidence re-ranking method
• Combines reliability, heterogeneity, and extrapolation analyses
• Filters PubMed evidence for improved RAG accuracy