Overview of TREC 2025 Biomedical Generative Retrieval (BioGen) Track

2026-03-23

AI Summary

This work addresses the challenge of hallucination in large language models when applied to biomedical generative retrieval by proposing a retrieval-augmented generation (RAG)-based, traceable generation framework. The approach was evaluated in the TREC 2025 BioGen track—the first benchmark dedicated to biomedical generative retrieval—employing novel evaluation metrics that emphasize evidential support and factual consistency to systematically assess alignment between generated content and authoritative scientific literature. Preliminary results reveal substantial deficiencies in the factual reliability of current models, underscoring the critical need for high-confidence, traceable biomedical generation systems. This study thus offers a promising pathway toward trustworthy AI applications in high-stakes clinical decision-making contexts.

Technology Category

• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems

Abstract

Recent advances in large language models (LLMs) have made significant progress across multiple biomedical tasks, including biomedical question answering, lay-language summarization of the biomedical literature, and clinical note summarization. These models have demonstrated strong capabilities in processing and synthesizing complex biomedical information and in generating fluent, human-like responses. Despite these advancements, hallucinations or confabulations remain key challenges when using LLMs in biomedical and other high-stakes domains. Inaccuracies may be particularly harmful in high-risk situations, such as medical question answering, making clinical decisions, or appraising biomedical research. Studies on the evaluation of the LLMs' abilities to ground generated statements in verifiable sources have shown that models perform significantly

Problem

• hallucination
• biomedical
• large language models
• factuality
• high-stakes domains

Innovation

• generative retrieval
• biomedical LLMs
• hallucination mitigation
• fact grounding
• verifiable generation