Diagnosing and Repairing Factual Errors in RAG under Budget Constraints

2026-06-28

AI Summary

This work addresses the prevalence of factual errors in deployed Retrieval-Augmented Generation (RAG) systems, which often stem from missing retrieval evidence or contextually inconsistent generation—issues that existing repair methods struggle to resolve under black-box conditions or resource constraints. To tackle this challenge, the authors propose D2R-RAG, a novel model-agnostic and resource-aware framework for diagnosing and repairing RAG failures. D2R-RAG employs lightweight modules to extract interpretable failure signatures from the query, retrieved passages, and generated response, then adaptively selects the optimal repair strategy under explicit latency and memory budgets. Experimental results demonstrate that D2R-RAG significantly enhances reliability on FEVER and HotpotQA, consistently outperforming existing baselines across diverse computational budgets while achieving a superior trade-off between accuracy and efficiency.

Abstract

Retrieval-Augmented Generation (RAG) improves the factuality of large language models by grounding responses in external evidence, yet real-world deployments remain fragile. Failures often stem from missing or weakly relevant evidence, as well as from generation that does not faithfully reflect the retrieved context. Many existing approaches rely on fine-tuning, privileged access to internal model signals, or resource-insensitive escalation strategies, which limits their practicality in black-box and budget-constrained settings. We propose D2R-RAG (Diagnose-to-Repair RAG), a model-agnostic and resource-aware framework that combines lightweight failure diagnosis with adaptive repair. D2R-RAG derives interpretable failure signatures from observable signals in the query, retrieved evidence, and generated response, and then selects from a small set of corrective actions under explicit latency and VRAM constraints. Experiments on FEVER and HotpotQA show that D2R-RAG improves reliability over recent baselines and achieves better accuracy--efficiency trade-offs across multiple compute budgets. The code is available at https://github.com/CyberScienceLab/D2R-RAG/.

Problem

• Retrieval-Augmented Generation
• factual errors
• budget constraints
• black-box setting
• failure diagnosis

Innovation

• RAG
• factuality
• budget-constrained
• model-agnostic
• adaptive repair