CFDA & CLIP at TREC iKAT 2025: Enhancing Personalized Conversational Search via Query Reformulation and Rank Fusion

2025-09-19

AI Summary

This work addresses the TREC iKAT 2025 task, tackling the joint optimization challenge among robustness and efficiency in real-time interactive conversational search, and accuracy in offline passage ranking and response generation. We propose a multi-stage retrieval-generation framework integrating query rewriting, Best-of-N response selection, and reciprocal rank fusion (RRF): lightweight rewriting and RRF enable low-latency, multi-source retrieval fusion during interaction; offline, passage re-ranking and Best-of-N response filtering enhance controllable accuracy. Our core innovation lies in unifying the modeling of interactive and offline requirements via an adaptive fusion mechanism that dynamically balances effectiveness and efficiency. Experiments demonstrate substantial improvements in system robustness and, for the first time, quantitatively characterize the effectiveness-efficiency trade-off boundary between interactive and offline evaluation paradigms.

Technology Category

• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Search and Optimization: Learning to Search
• Natural Language Processing: Generation

Application Category

• Search and Retrieval-Augmented AI: Assisted, interactive, and conversational search
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

The 2025 TREC Interactive Knowledge Assistance Track (iKAT) featured both interactive and offline submission tasks. The former requires systems to operate under real-time constraints, making robustness and efficiency as important as accuracy, while the latter enables controlled evaluation of passage ranking and response generation with pre-defined datasets. To address this, we explored query rewriting and retrieval fusion as core strategies. We built our pipelines around Best-of-$N$ selection and Reciprocal Rank Fusion (RRF) strategies to handle different submission tasks. Results show that reranking and fusion improve robustness while revealing trade-offs between effectiveness and efficiency across both tasks.

Problem

• Enhancing personalized conversational search accuracy
• Improving robustness and efficiency in real-time systems
• Optimizing query reformulation and rank fusion strategies

Innovation

• Query rewriting for search enhancement
• Reciprocal Rank Fusion for retrieval
• Best-of-N selection strategy