POLYRAG: Integrating Polyviews into Retrieval-Augmented Generation for Medical Applications

2025-04-21

AI Summary

Large language models (LLMs) in healthcare suffer from outdated knowledge and factual hallucinations; existing retrieval-augmented generation (RAG) methods neglect temporal freshness, source authority, and inter-source consensus, failing to resolve multi-source conflicts and time-varying information. Method: We propose PolyRAG, a multi-perspective RAG framework featuring (i) perspective-aware retrieval, (ii) weighted view fusion, and (iii) a multi-label (timeliness/authority/consensus)-driven evidence evaluation and aggregation mechanism. Contribution/Results: We introduce PolyEVAL—the first multidimensionally annotated benchmark tailored to real-world clinical scenarios—and empirically demonstrate that PolyRAG significantly outperforms state-of-the-art RAG baselines on PolyEVAL, achieving substantial gains in answer accuracy and clinical credibility while effectively mitigating decision biases induced by conflicting evidence.

Technology Category

• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty
• Machine Learning: Large Multimodal Models (LMMs)
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Large language models (LLMs) have become a disruptive force in the industry, introducing unprecedented capabilities in natural language processing, logical reasoning and so on. However, the challenges of knowledge updates and hallucination issues have limited the application of LLMs in medical scenarios, where retrieval-augmented generation (RAG) can offer significant assistance. Nevertheless, existing retrieve-then-read approaches generally digest the retrieved documents, without considering the timeliness, authoritativeness and commonality of retrieval. We argue that these approaches can be suboptimal, especially in real-world applications where information from different sources might conflict with each other and even information from the same source in different time scale might be different, and totally relying on this would deteriorate the performance of RAG approaches. We propose PolyRAG that carefully incorporate judges from different perspectives and finally integrate the polyviews for retrieval augmented generation in medical applications. Due to the scarcity of real-world benchmarks for evaluation, to bridge the gap we propose PolyEVAL, a benchmark consists of queries and documents collected from real-world medical scenarios (including medical policy, hospital&doctor inquiry and healthcare) with multiple tagging (e.g., timeliness, authoritativeness) on them. Extensive experiments and analysis on PolyEVAL have demonstrated the superiority of PolyRAG.

Problem

• Addressing knowledge update and hallucination issues in medical LLMs
• Improving retrieval quality by considering timeliness, authoritativeness, and commonality
• Resolving conflicts from multi-source medical information in RAG systems

Innovation

• Integrates polyviews for medical RAG
• Considers timeliness and authoritativeness in retrieval
• Introduces PolyEVAL benchmark for evaluation