From Voting to Agent Collaboration: Answer-Type-Aware LLM Pipelines for BioASQ 14b

2026-07-07

AI Summary

This work proposes a question-type-aware reasoning framework for large language models (LLMs) tailored to the distinct inferential demands of biomedical question answering, specifically addressing yes/no, factoid, and list-type questions. The approach integrates chain-of-thought prompting, in-context learning, and a self-reflection mechanism, while introducing a multi-agent collaborative reasoning pipeline for list questions that enhances answer robustness and traceability through passage re-ranking and ensemble prediction. Evaluated on Task B of BioASQ 14b, the system achieved top performance on the factoid subtask in Batch 4 and demonstrated competitive overall results.

Abstract

Biomedical question answering requires not only accurate extraction of information from scientific literature but also reliable integration of evidence across multiple documents. This study presents a question-type-specific large language model (LLM) framework for BioASQ 14b Task B, designed to improve answer robustness and evidence grounding in biomedical question answering. Rather than applying a single prompting strategy to all questions, the framework selects different inference procedures for yes/no, factoid, and list questions according to their distinct reasoning and evaluation requirements. For yes/no questions, snippet shuffling and self-reflection are used to reduce sensitivity to evidence ordering and improve decision stability. For factoid questions, full-snippet input is combined with chain-of-thought-based in-context learning to support accurate biomedical entity identification. For list questions, a multi-agent architecture is employed, in which evidence extraction, candidate generation, answer verification, and final aggregation are handled collaboratively. Preliminary experiments on BioASQ 13b were used to identify effective inference strategies for each question type, and the resulting framework was subsequently evaluated in the official BioASQ 14b Task B challenge. In the official evaluation, our framework showed competitive performance across multiple batches and achieved first place in the factoid subtask of Batch 4. These results demonstrate the effectiveness of combining question-type-specific inference, ensemble prediction, and agent-based verification for reliable biomedical question answering.

Problem

• biomedical question answering
• question-type-specific
• evidence integration
• answer robustness
• BioASQ

Innovation

• question-type-specific LLM
• multi-agent collaboration
• chain-of-thought reasoning
• evidence grounding
• self-reflection