Overview of the TREC 2025 Retrieval Augmented Generation (RAG) Track

2026-03-10

AI Summary

This work proposes a retrieval-augmented generation (RAG) evaluation framework tailored to deep informational needs, addressing the challenges of reasoning and factual consistency posed by multi-sentence narrative queries in complex real-world scenarios. The framework introduces narrative-style multi-sentence queries and a multi-level evaluation protocol, enabling comprehensive assessment of system performance in terms of completeness, attributability, and consistency over the MS MARCO V2.1 corpus. Aggregating over 150 system submissions, the study demonstrates the feasibility of simultaneously ensuring factual accuracy, transparency, and high-quality generation under complex querying conditions, thereby establishing a new benchmark for trustworthy, context-aware RAG systems.

Technology Category

• Natural Language Processing: Question Answering
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Knowledge Representation and Reasoning: Common-Sense Reasoning

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

The second edition of the TREC Retrieval Augmented Generation (RAG) Track advances research on systems that integrate retrieval and generation to address complex, real-world information needs. Building on the foundation of the inaugural 2024 track, this year's challenge introduces long, multi-sentence narrative queries to better reflect the deep search task with the growing demand for reasoning-driven responses. Participants are tasked with designing pipelines that combine retrieval and generation while ensuring transparency and factual grounding. The track leverages the MS MARCO V2.1 corpus and employs a multi-layered evaluation framework encompassing relevance assessment, response completeness, attribution verification, and agreement analysis. By emphasizing multi-faceted narratives and attribution-rich answers from over 150 submissions this year, the TREC 2025 RAG Track aims to foster innovation in creating trustworthy, context-aware systems for retrieval augmented generation.

Problem

• Retrieval Augmented Generation
• complex information needs
• reasoning-driven responses
• factual grounding
• narrative queries

Innovation

• Retrieval Augmented Generation
• narrative queries
• attribution verification
• multi-layered evaluation
• factual grounding