PrismRAG: Boosting RAG Factuality with Distractor Resilience and Strategized Reasoning

2025-07-24

AI Summary

RAG systems often produce factual inaccuracies when confronted with semantically distracting passages or complex questions requiring deep reasoning. To address this, we propose a distractor-aware efficient fine-tuning framework. Our method introduces three key components: (1) distractor-aware question-answer contrastive learning pairs to explicitly model robustness against irrelevant context; (2) a prompt-based reasoning trajectory guidance mechanism that enforces structured, multi-step inference; and (3) end-to-end joint optimization that autonomously elicits planning and synthesis capabilities without handcrafted instructions. Evaluated on 12 open-domain RAG benchmarks, our approach achieves an average 5.4% improvement in factual accuracy over prior state-of-the-art methods. It establishes a scalable, low-intervention paradigm for enhancing factual consistency in RAG—requiring neither external modules nor manual annotation—while preserving efficiency and generalizability.

Technology Category

• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Knowledge Representation and Reasoning: Argumentation
• Search and Optimization: Distributed Search

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Retrieval-augmented generation (RAG) often falls short when retrieved context includes confusing semi-relevant passages, or when answering questions require deep contextual understanding and reasoning. We propose an efficient fine-tuning framework, called PrismRAG, that (i) trains the model with distractor-aware QA pairs mixing gold evidence with subtle distractor passages, and (ii) instills reasoning-centric habits that make the LLM plan, rationalize, and synthesize without relying on extensive human engineered instructions. Evaluated across 12 open-book RAG QA benchmarks spanning diverse application domains and scenarios, PrismRAG improves average factuality by 5.4%, outperforming state-of-the-art solutions.

Problem

• Enhancing RAG factuality with distractor resilience
• Improving deep contextual understanding and reasoning
• Reducing reliance on human-engineered instructions

Innovation

• Fine-tuning with distractor-aware QA pairs
• Instilling reasoning-centric model habits
• Improving factuality without human instructions