MedJudgeRAG: Option-Wise Evidence Judgment with Dynamic Knowledge Graphs for Medical MCQA

2026-07-24

AI Summary

This work addresses the limitations of conventional retrieval-augmented generation (RAG) in medical multiple-choice question answering (MCQA), where indiscriminate use of retrieved documents often introduces noise and degrades model performance. The authors propose a novel reasoning framework that integrates dynamic knowledge graphs with option-level evidence evaluation. Specifically, the approach independently assesses the relevance of retrieved evidence for each answer option, dynamically selects an appropriate knowledge utilization strategy based on this assessment, and employs graph-conditioned supervised training to refine structured reasoning trajectories. Evaluated on two medical MCQA benchmarks, the method significantly outperforms both standard RAG and parametric baselines, demonstrating that dynamic knowledge graphs can effectively guide knowledge integration during training.

Abstract

In medical multiple-choice question answering (MCQA), Retrieval-Augmented Generation (RAG) can supplement the domain knowledge of language models (LMs). However, since vanilla RAG indiscriminately utilizes retrieved documents, it can degrade LM performance. To address this, we propose MedJudgeRAG. Our framework represents retrieved documents as a dynamic knowledge graph (KG) composed of entities and relations. For each option, the model judges an evidence verdict from the retrieved documents and the KG. Based on the verdict combination, the model determines a knowledge utilization strategy to reason toward the final answer. These capabilities are trained via supervised fine-tuning using structured reasoning traces generated by a teacher LM. The training employs a weighted cross-entropy loss that differentially weights the KG and reasoning segments. Experiments on two medical MCQA benchmarks demonstrate that MedJudgeRAG consistently outperforms both vanilla RAG and parametric baselines. Furthermore, ablation analysis reveals that the dynamic KG is more effective as graph-conditioned supervision at training time than as an explicit output at inference time. Our code is available at https://github.com/hyu-amllab/medjudgerag, and the generated reasoning traces are released at https://huggingface.co/datasets/youarethewon/medjudgerag.

Problem

• Medical MCQA
• Retrieval-Augmented Generation
• Evidence Judgment
• Knowledge Utilization
• Language Models

Innovation

• Retrieval-Augmented Generation
• Dynamic Knowledge Graph
• Evidence Judgment
• Medical MCQA
• Supervised Fine-tuning