TOBUGraph: Knowledge Graph-Based Retrieval for Enhanced LLM Performance Beyond RAG

2024-12-06

AI Summary

Retrieval-Augmented Generation (RAG) suffers from inherent limitations, including semantic fragmentation, sensitivity to text chunking, and frequent hallucinations. To address these issues, we propose TOBUGraph—the first end-to-end LLM-driven dynamic knowledge graph retrieval framework. It leverages large language models to automatically extract entities and relations from unstructured text, constructing an evolvable knowledge graph. Crucially, it replaces vector-similarity-based matching with semantic-aware graph traversal, enabling deep cross-chunk relational retrieval and eliminating reliance on manual text segmentation. Evaluated within the production system TOBU, TOBUGraph significantly outperforms multiple RAG baselines in precision and recall, while substantially improving user retrieval satisfaction and task completion rates. This work establishes a semantics-native pathway for retrieval-augmented generation, advancing beyond traditional embedding-centric paradigms.

Technology Category

• Data Mining & Knowledge Management: Linked Open Data, Knowledge Graphs & KB Completion
• Machine Learning: Graph-based Machine Learning
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Retrieval-Augmented Generation (RAG) is one of the leading and most widely used techniques for enhancing LLM retrieval capabilities, but it still faces significant limitations in commercial use cases. RAG primarily relies on the query-chunk text-to-text similarity in the embedding space for retrieval and can fail to capture deeper semantic relationships across chunks, is highly sensitive to chunking strategies, and is prone to hallucinations. To address these challenges, we propose TOBUGraph, a graph-based retrieval framework that first constructs the knowledge graph from unstructured data dynamically and automatically. Using LLMs, TOBUGraph extracts structured knowledge and diverse relationships among data, going beyond RAG's text-to-text similarity. Retrieval is achieved through graph traversal, leveraging the extracted relationships and structures to enhance retrieval accuracy, eliminating the need for chunking configurations while reducing hallucination. We demonstrate TOBUGraph's effectiveness in TOBU, a real-world application in production for personal memory organization and retrieval. Our evaluation using real user data demonstrates that TOBUGraph outperforms multiple RAG implementations in both precision and recall, significantly improving user experience through improved retrieval accuracy.

Problem

• Enhances LLM retrieval beyond RAG limitations
• Constructs dynamic knowledge graphs from unstructured data
• Improves retrieval accuracy and reduces hallucinations

Innovation

• Dynamic knowledge graph construction from unstructured data
• Graph traversal for enhanced retrieval accuracy
• Eliminates chunking, reduces hallucination in LLMs