ConflictRAG: Detecting and Resolving Knowledge Conflicts in Retrieval Augmented Generation

2026-05-17

AI Summary

This work addresses the challenge of reduced answer accuracy in traditional retrieval-augmented generation (RAG) systems caused by knowledge conflicts among retrieved documents. To enhance system reliability, the authors propose ConflictRAG, a framework that detects, classifies, and resolves conflicts prior to generation. The approach features a lightweight two-stage conflict detection mechanism, a data-driven source credibility assessment based on entropy and TOPSIS, and a novel conflict-aware CARS evaluation metric. Experimental results demonstrate that ConflictRAG achieves an F1 score of 88.7% for conflict detection across three benchmarks, yielding a 5.3–6.1% improvement in answer correctness over the strongest baseline while reducing API invocation costs by 62%.

Abstract

Retrieval-Augmented Generation (RAG) systems implicitly assume mutual consistency among retrieved documents -- an assumption that frequently fails in practice. We present ConflictRAG, a conflict-aware RAG framework that detects, classifies, and resolves knowledge conflicts prior to answer generation. The framework introduces three contributions: (1) a two-stage conflict detection module combining a lightweight embedding-based MLP classifier with selective LLM refinement, reducing API costs by 62% while maintaining 90.8% detection accuracy; (2) an Entropy-TOPSIS framework for data-driven source credibility assessment, improving selection accuracy by 7.1% over manual heuristics; and (3) a Conflict-Aware RAG Score (CARS) for diagnostic evaluation of conflict-handling capabilities. Experiments on three benchmarks against six baselines demonstrate 88.7% conflict-detection F1 and consistent 5.3--6.1% correctness gains over the strongest conflict-aware baseline, with the pipeline transferring effectively across backbone LLMs.

Problem

• Retrieval-Augmented Generation
• knowledge conflicts
• conflict detection
• source credibility
• RAG systems

Innovation

• Conflict Detection
• Retrieval-Augmented Generation
• Source Credibility
• Entropy-TOPSIS
• CARS