Reasoning of Large Language Models over Knowledge Graphs with Super-Relations

2025-03-28

AI Summary

Existing large language models (LLMs) suffer from low multi-hop relation retrieval rates and insufficient coverage in knowledge graph question answering (KGQA). To address this, we propose ReKnoS, a novel framework introducing the concept of “hyper-relations”—abstracting multi-hop paths into unified semantic units to enable joint forward and backward reasoning, thereby overcoming the limitations of conventional greedy search. ReKnoS integrates knowledge graph embedding, path pattern induction, and LLM prompt enhancement to jointly realize relation aggregation, bidirectional inference scheduling, and efficient retrieval. Evaluated on nine real-world datasets, ReKnoS achieves an average 2.92% improvement in question-answering accuracy and significantly outperforms state-of-the-art methods in successful retrieval rate. These results demonstrate that hyper-relation modeling critically expands the effective search space and enhances reasoning robustness in KGQA.

Technology Category

• Knowledge Representation and Reasoning: Knowledge Acquisition
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Search and Optimization: Learning to Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Large language models for search
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

While large language models (LLMs) have made significant progress in processing and reasoning over knowledge graphs, current methods suffer from a high non-retrieval rate. This limitation reduces the accuracy of answering questions based on these graphs. Our analysis reveals that the combination of greedy search and forward reasoning is a major contributor to this issue. To overcome these challenges, we introduce the concept of super-relations, which enables both forward and backward reasoning by summarizing and connecting various relational paths within the graph. This holistic approach not only expands the search space, but also significantly improves retrieval efficiency. In this paper, we propose the ReKnoS framework, which aims to Reason over Knowledge Graphs with Super-Relations. Our framework's key advantages include the inclusion of multiple relation paths through super-relations, enhanced forward and backward reasoning capabilities, and increased efficiency in querying LLMs. These enhancements collectively lead to a substantial improvement in the successful retrieval rate and overall reasoning performance. We conduct extensive experiments on nine real-world datasets to evaluate ReKnoS, and the results demonstrate the superior performance of ReKnoS over existing state-of-the-art baselines, with an average accuracy gain of 2.92%.

Problem

• Improving retrieval rates for LLMs over knowledge graphs
• Enhancing reasoning with forward-backward super-relations
• Boosting accuracy in graph-based question answering

Innovation

• Introducing super-relations for bidirectional reasoning
• ReKnoS framework enhances retrieval efficiency
• Combines multiple relation paths for accuracy