5ting at SemEval-2026 Task 8: Strong End-to-End Multi-Turn RAG via LLM-Based Reranking and Faithfulness Control

2026-06-27

AI Summary

This work addresses critical challenges in multi-turn retrieval-augmented generation (RAG)—namely, context drift, query ambiguity, and hallucination risks—by proposing an end-to-end system that integrates dual-query dense retrieval (leveraging BGE-M3 and FAISS), large language model–based re-ranking, and a role-separated generation mechanism grounded strictly in retrieved evidence to enforce output faithfulness. Evaluated on SemEval-2026 Task 8, the approach substantially mitigates key limitations of multi-turn RAG, achieving a retriever nDCG@5 of 0.4719 and, in Task C, a harmonic mean score of 0.5597 alongside an RL_F score of 0.7692, thereby demonstrating both its effectiveness and methodological novelty.

Abstract

We introduce 5ting, our system for the SemEval2026 Task 8 (MTRAGEval), which evaluates multi-turn Retrieval Augmented Generation (RAG) systems. Multi turn RAG involves context drift, under specification, and hallucination risk. Our system combines BGE-M3 dense retrieval with FAISS indexing, dual-query merged retrieval, and LLM based reranking, followed by role separated generation constrained to retrieved evidence. The retriever achieved nDCG@5 = 0.4719 in Task A, while the end to end system ranked in Task C with a harmonic score of 0.5597 and RL_F = 0.7692.

Problem

• multi-turn RAG
• context drift
• under-specification
• hallucination

Innovation

• Multi-turn RAG
• LLM-based reranking
• Faithfulness control
• Dual-query retrieval
• Role-separated generation