Weaving Multi-Source Evidence for Biomedical Reasoning: The BioMedHop Benchmark and BioWeave Framework

2026-06-15

AI Summary

Existing biomedical question-answering benchmarks lack support for multi-source graph reasoning across knowledge graphs, literature, and web resources, as well as structured evidence topology. To address this gap, this work introduces the BioMedHop benchmark and the BioWeave framework. BioMedHop is the first benchmark to systematically support multi-source, multi-type reasoning with diverse answer formats, while BioWeave enables structured reasoning and verification through knowledge graph path retrieval, multi-source information fusion, and evidence graph construction. Experimental results demonstrate that BioWeave outperforms the strong baseline ToG-2 by 10.5% in overall performance on BioMedHop and significantly enhances the reasoning capabilities of large language models across different scales, with smaller models achieving performance comparable to GPT-4-Turbo.

Abstract

Biomedical question answering (QA) increasingly requires reasoning over interacting entities, where supporting evidence is scattered across biomedical knowledge graphs, literature documents, and web-accessible resources. However, existing biomedical QA benchmarks mainly focus on exam-style knowledge, literature comprehension, or short-range multi-hop inference, leaving source-conditioned graph reasoning and evidence topology construction underexplored. To fill this gap, we introduce BioMedHop, a multi-source graph-grounded benchmark for evaluating biomedical reasoning over structured evidence topologies. BioMedHop contains 10,045 instances across KG, document, web, and hybrid evidence settings, covering shared-neighbor matching, intersection reasoning, path-based reasoning, and counting, with option-based, open-ended, and numeric count renderings. To support this benchmark, we further propose BioWeave, a source-aware reasoning framework that retrieves biomedical KG paths, gathers supporting clues from documents and web sources, assembles them into a unified evidence graph, and verifies answers through entity-level evidence support. Comprehensive experiments show that BioWeave achieves the best overall performance among compared methods on BioMedHop, outperforming the strong hybrid baseline ToG-2 by 10.5% in the overall average. Moreover, BioWeave consistently improves different LLM backbones and enables smaller models, such as Qwen3-4B, to achieve reasoning performance comparable to GPT-4-Turbo.

Problem

• biomedical question answering
• multi-source reasoning
• evidence topology
• knowledge graph
• multi-hop inference

Innovation

• multi-source reasoning
• evidence graph
• biomedical QA
• BioMedHop
• BioWeave