Overview of the TREC 2025 Tip-of-the-Tongue track

2026-01-28

AI Summary

This work addresses the known-item retrieval challenge posed by the Tip-of-the-Tongue (ToT) phenomenon, where users struggle to accurately recall the identifier of a target item. For the first time, the ToT retrieval task is extended beyond narrow domains into general-purpose settings. The study constructs a more realistic and challenging ad hoc retrieval benchmark by integrating multiple test sources, including the MS-ToT dataset, human-curated topics, and synthetic queries generated by large language models. Employing a standard information retrieval evaluation framework, the initiative attracted 9 participating teams submitting 32 system runs, yielding a rich set of baselines and empirical results. This effort significantly advances research on retrieval systems under complex, naturally expressed queries that reflect real-world user difficulties in articulating precise search intents.

Technology Category

• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Natural Language Processing: Information Extraction
• Computer Vision: Image and Video Retrieval

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Querying, indexing, and retrieval in Web-related graphs

Abstract

Tip-of-the-tongue (ToT) known-item retrieval involves re-finding an item for which the searcher does not reliably recall an identifier. ToT information requests (or queries) are verbose and tend to include several complex phenomena, making them especially difficult for existing information retrieval systems. The TREC 2025 ToT track focused on a single ad-hoc retrieval task. This year, we extended the track to general domain and incorporated different sets of test queries from diverse sources, namely from the MS-ToT dataset, manual topic development, and LLM-based synthetic query generation. This year, 9 groups (including the track coordinators) submitted 32 runs.

Problem

• Tip-of-the-tongue
• known-item retrieval
• information retrieval
• complex queries
• ad-hoc retrieval

Innovation

• Tip-of-the-tongue retrieval
• ad-hoc retrieval
• synthetic query generation
• large language models
• multi-source query collection