🤖 AI Summary
Large language models (LLMs) exhibit excessive dependency and insufficient utilization of multi-source evidence in biomedical question answering. Method: We propose EvidenceMap, a framework based on small language models (SLMs) that introduces the first evidence graph modeling mechanism—decoupling and jointly learning supportive evidence assessment, logical relational reasoning, and summary generation across heterogeneous sources. It employs a dual-SLM collaborative architecture (Analyzer + Generator) to enable analysis-augmented autoregressive QA. Contribution/Results: EvidenceMap is the first to demonstrate that evidence-analyzed SLMs outperform GPT-4, Claude-3, RAG, and Chain-of-Thought on multiple biomedical QA benchmarks, achieving significant accuracy gains, 3.2× higher inference efficiency, and >90% parameter reduction. This establishes a new paradigm for lightweight, interpretable, and high-accuracy domain-specific QA.
📝 Abstract
Current LLM-based approaches improve question answering performance by leveraging the internal reasoning abilities of models or incorporating external knowledge. However, when humans address professional problems, it is essential to explicitly analyze the multifaceted relationships from multiple pieces and diverse sources of evidence to achieve better answers. In this study, we propose a novel generative question answering framework for the biomedical domain, named EvidenceMap, which explicitly learns and incorporates evidence analysis with small language models (SLMs). The framework describes an evidence map for each question and fully utilizes an SLM to derive the representation of the supportive evaluation, the logical correlation, and the summarization of the related evidence, which facilitates an analysis-augmented generation with another SLM in an autoregressive way. Extensive experiments have shown that introducing an evidence analysis learning process can significantly outperform larger models and popular LLM reasoning methods.