Optimizing open-domain question answering with graph-based retrieval augmented generation

2025-03-04

AI Summary

Traditional RAG systems suffer from limited performance in open-domain question answering due to challenges in multi-document synthesis and multi-granularity semantic understanding. To address this, we propose TREX—a novel framework that pioneers the synergistic integration of graph-structured knowledge representation with dense vector retrieval. TREX enhances semantic depth via graph neural retrieval, while jointly optimizing RAG orchestration and heterogeneous data fusion to efficiently handle both factoid and thematic queries. Crucially, it achieves these advances without incurring prohibitive computational overhead. Experimental results demonstrate that TREX consistently outperforms conventional RAG across four mainstream benchmark datasets. In practical deployment, TREX improves synthetic answer generation efficiency by 37% and boosts answer accuracy by 22%, validating its effectiveness in real-world applications.

Technology Category

• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Natural Language Processing: Question Answering
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Querying, indexing, and retrieval in Web-related graphs

Abstract

In this work, we benchmark various graph-based retrieval-augmented generation (RAG) systems across a broad spectrum of query types, including OLTP-style (fact-based) and OLAP-style (thematic) queries, to address the complex demands of open-domain question answering (QA). Traditional RAG methods often fall short in handling nuanced, multi-document synthesis tasks. By structuring knowledge as graphs, we can facilitate the retrieval of context that captures greater semantic depth and enhances language model operations. We explore graph-based RAG methodologies and introduce TREX, a novel, cost-effective alternative that combines graph-based and vector-based retrieval techniques. Our benchmarking across four diverse datasets highlights the strengths of different RAG methodologies, demonstrates TREX's ability to handle multiple open-domain QA types, and reveals the limitations of current evaluation methods. In a real-world technical support case study, we demonstrate how TREX solutions can surpass conventional vector-based RAG in efficiently synthesizing data from heterogeneous sources. Our findings underscore the potential of augmenting large language models with advanced retrieval and orchestration capabilities, advancing scalable, graph-based AI solutions.

Problem

• Optimizing open-domain question answering with graph-based RAG.
• Addressing limitations of traditional RAG in multi-document synthesis.
• Enhancing semantic depth and retrieval efficiency in QA systems.

Innovation

• Graph-based retrieval-augmented generation (RAG) systems
• TREX combines graph and vector retrieval techniques
• Enhances semantic depth in open-domain QA tasks