ContraGen: A Multi-Agent Generation Framework for Enterprise Contradictions Detection

2025-10-03

AI Summary

Enterprise RAG systems frequently produce untrustworthy outputs due to intrinsic or cross-document contradictions in external documents—yet existing contradiction detection benchmarks operate only at the sentence level and fail to capture the structural and semantic complexity of high-stakes enterprise documents (e.g., contracts, financial reports). Method: We propose the first enterprise-oriented contradiction detection benchmark framework. It introduces a fine-grained, enterprise-specific contradiction taxonomy supporting controllable synthesis of self-consistency and mutual-exclusivity contradictions; integrates multi-agent generation, domain-knowledge injection, human-in-the-loop reinforcement learning, and retrieval consistency evaluation for high-fidelity document synthesis; and establishes a human–AI collaborative verification pipeline for end-to-end contradiction-aware evaluation. Contribution/Results: Experiments demonstrate that our framework effectively detects multi-granularity contradictions in RAG outputs, significantly enhancing system trustworthiness and explainability in high-risk domains such as regulatory compliance and corporate governance.

Technology Category

• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Multiagent Systems: Adversarial Agents
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Retrieval-Augmented Generation (RAG) integrates LLMs with external sources, offering advanced capabilities for information access and decision-making. However, contradictions in retrieved evidence can result in inconsistent or untrustworthy outputs, which is especially problematic in enterprise settings where compliance, governance, and accountability are critical. Existing benchmarks for contradiction detection are limited to sentence-level analysis and do not capture the complexity of enterprise documents such as contracts, financial filings, compliance reports, or policy manuals. To address this limitation, we propose ContraGen, a contradiction-aware benchmark framework tailored to enterprise domain. The framework generates synthetic enterprise-style documents with embedded contradictions, enabling systematic evaluation of both intra-document and cross-document consistency. Automated contradiction mining is combined with human-in-the-loop validation to ensure high accuracy. Our contributions include generating realistic enterprise documents, modeling a taxonomy of contradiction types common in business processes, enabling controlled creation of self- and pairwise contradictions, developing a contradiction-aware retrieval evaluation pipeline and embedding human oversight to reflect domain-specific judgment complexity. This work establishes a foundation for more trustworthy and accountable RAG systems in enterprise information-seeking applications, where detecting and resolving contradictions is essential for reducing risk and ensuring compliance.

Problem

• Detecting contradictions in enterprise documents for compliance
• Addressing limitations of sentence-level contradiction detection benchmarks
• Generating synthetic enterprise documents with embedded contradictions

Innovation

• Generates synthetic enterprise documents with contradictions
• Models taxonomy of business process contradiction types
• Combines automated mining with human validation