DS@GT at TREC TOT 2025: Bridging Vague Recollection with Fusion Retrieval and Learned Reranking

2026-01-21

AI Summary

This work addresses the challenge of retrieving relevant documents under fuzzy memory conditions in the TREC Tip-of-the-Tongue task by proposing a two-stage retrieval framework. In the first stage, a topic-aware multi-index dense retrieval strategy is introduced, which integrates query representations from BM25, BGE-M3, and large language model (LLM)-generated queries, leveraging partitioned indexes built over 24 Wikipedia topic domains. The second stage employs a LambdaMART learning-to-rank model combined with a Gemini-2.5-flash reranker, trained on 5,000 LLM-synthesized queries. By effectively fusing multi-source retrieval signals with the reranking power of large language models, the proposed approach achieves a recall of 0.66 and an NDCG@1000 of 0.41 on the test set, substantially improving retrieval performance for ambiguous queries.

Technology Category

• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Querying, indexing, and retrieval in Web-related graphs

Abstract

We develop a two-stage retrieval system that combines multiple complementary retrieval methods with a learned reranker and LLM-based reranking, to address the TREC Tip-of-the-Tongue (ToT) task. In the first stage, we employ hybrid retrieval that merges LLM-based retrieval, sparse (BM25), and dense (BGE-M3) retrieval methods. We also introduce topic-aware multi-index dense retrieval that partitions the Wikipedia corpus into 24 topical domains. In the second stage, we evaluate both a trained LambdaMART reranker and LLM-based reranking. To support model training, we generate 5000 synthetic ToT queries using LLMs. Our best system achieves recall of 0.66 and NDCG@1000 of 0.41 on the test set by combining hybrid retrieval with Gemini-2.5-flash reranking, demonstrating the effectiveness of fusion retrieval.

Problem

• Tip-of-the-Tongue
• vague recollection
• information retrieval
• fusion retrieval
• TREC ToT

Innovation

• fusion retrieval
• topic-aware dense retrieval
• LLM-based reranking
• synthetic query generation
• hybrid retrieval