Rag Performance Prediction for Question Answering

2026-04-09

AI Summary

This study addresses the challenge of accurately predicting the performance gain of Retrieval-Augmented Generation (RAG) over non-RAG approaches in question-answering tasks. To overcome limitations of existing prediction methods, the authors propose a novel supervised post-generation predictor that explicitly models the semantic relationships among the input question, retrieved passages, and the generated answer. The approach integrates multi-dimensional signals from pre-retrieval, post-retrieval, and post-generation stages and is optimized through end-to-end training. Experimental results demonstrate that the proposed method significantly outperforms current prediction strategies across multiple benchmarks, offering a reliable basis for deciding whether to activate RAG in practical systems.

Abstract

We address the task of predicting the gain of using RAG (retrieval augmented generation) for question answering with respect to not using it. We study the performance of a few pre-retrieval and post-retrieval predictors originally devised for ad hoc retrieval. We also study a few post-generation predictors, one of which is novel to this study and posts the best prediction quality. Our results show that the most effective prediction approach is a novel supervised predictor that explicitly models the semantic relationships among the question, retrieved passages, and the generated answer.

Problem

• RAG
• performance prediction
• question answering
• retrieval augmented generation

Innovation

• RAG performance prediction
• retrieval-augmented generation
• semantic relationship modeling
• supervised predictor
• question answering