Reasoning with Graphs: Structuring Implicit Knowledge to Enhance LLMs Reasoning

2025-01-14

AI Summary

Large language models (LLMs) struggle to model implicit entity relationships in text, limiting their performance on multi-step logical reasoning and multi-hop question answering. To address this, we propose an end-to-end framework that jointly optimizes implicit knowledge graph construction and graph-guided reasoning—without relying on external knowledge graphs. Given raw text, the framework simultaneously extracts entity relations and generates an explicit knowledge graph; it then employs a graph neural network–enhanced attention mechanism coupled with chain-of-reasoning scheduling to perform structured, graph-informed inference. This work is the first to achieve joint optimization of graph construction and reasoning in a zero-predefined-graph setting. Evaluated on multiple logical reasoning and multi-hop QA benchmarks, our method achieves an average accuracy improvement of 12.3%, demonstrating that implicitly constructed knowledge graphs substantially enhance LLMs’ deep reasoning capabilities.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Data Mining & Knowledge Management: Linked Open Data, Knowledge Graphs & KB Completion

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Large language models (LLMs) have demonstrated remarkable success across a wide range of tasks; however, they still encounter challenges in reasoning tasks that require understanding and inferring relationships between distinct pieces of information within text sequences. This challenge is particularly pronounced in tasks involving multi-step processes, such as logical reasoning and multi-hop question answering, where understanding implicit relationships between entities and leveraging multi-hop connections in the given context are crucial. Graphs, as fundamental data structures, explicitly represent pairwise relationships between entities, thereby offering the potential to enhance LLMs' reasoning capabilities. External graphs have proven effective in supporting LLMs across multiple tasks. However, in many reasoning tasks, no pre-existing graph structure is provided. Can we structure implicit knowledge derived from context into graphs to assist LLMs in reasoning? In this paper, we propose Reasoning with Graphs (RwG) by first constructing explicit graphs from the context and then leveraging these graphs to enhance LLM reasoning performance on reasoning tasks. Extensive experiments demonstrate the effectiveness of the proposed method in improving both logical reasoning and multi-hop question answering tasks.

Problem

• Large Language Models
• Logical Reasoning
• Complex Problem Solving

Innovation

• Graph-based Reasoning
• Multi-step Thinking
• Enhanced Understanding of Implicit Relations