OpenDecoder: Open Large Language Model Decoding to Incorporate Document Quality in RAG

2026-01-13

AI Summary

This work proposes OpenDecoder, a novel framework that addresses the challenge of inconsistent retrieval quality in Retrieval-Augmented Generation (RAG) systems, which often undermines answer accuracy. OpenDecoder is the first approach to explicitly integrate multi-dimensional document quality signals—including relevance scores, ranking positions, and query performance prediction metrics—directly into the decoding process of large language models (LLMs), enabling quality-aware generation control. The framework is highly flexible, allowing seamless incorporation of arbitrary external quality indicators and compatibility with various LLM post-training objectives. Extensive experiments across five benchmark datasets demonstrate that OpenDecoder significantly outperforms existing baselines, substantially enhancing the robustness of RAG systems against noisy contexts and improving the reliability of generated responses.

Technology Category

• Knowledge Representation and Reasoning: Qualitative Reasoning
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Question Answering

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

The development of large language models (LLMs) has achieved superior performance in a range of downstream tasks, including LLM-based retrieval-augmented generation (RAG). The quality of generated content heavily relies on the usefulness of the retrieved information and the capacity of LLMs'internal information processing mechanism to incorporate it in answer generation. It is generally assumed that the retrieved information is relevant to the question. However, the retrieved information may have a variable degree of relevance and usefulness, depending on the question and the document collection. It is important to take into account the relevance of the retrieved information in answer generation. In this paper, we propose OpenDecoder, a new approach that leverages explicit evaluation of the retrieved information as quality indicator features for generation. We aim to build a RAG model that is more robust to varying levels of noisy context. Three types of explicit evaluation information are considered: relevance score, ranking score, and QPP (query performance prediction) score. The experimental results on five benchmark datasets demonstrate the effectiveness and better robustness of OpenDecoder by outperforming various baseline methods. Importantly, this paradigm is flexible to be integrated with the post-training of LLMs for any purposes and incorporated with any type of external indicators.

Problem

• retrieval-augmented generation
• document quality
• relevance evaluation
• noisy context
• LLM robustness

Innovation

• OpenDecoder
• retrieval-augmented generation
• document quality
• query performance prediction
• LLM decoding