From Query to Logic: Ontology-Driven Multi-Hop Reasoning in LLMs

2025-08-02

AI Summary

Large language models (LLMs) struggle with modeling nonlinear, structured reasoning in multi-hop question answering (MQA). Method: This paper proposes ORACLE, a novel framework that dynamically constructs question-oriented knowledge ontologies and automatically compiles them into first-order logic (FOL) reasoning chains—thereby integrating the structural expressiveness of knowledge graphs with the semantic understanding capability of LLMs. ORACLE employs an LLM-driven pipeline for ontology construction, logical formalization, and subproblem decomposition in concert. Contribution/Results: The framework significantly enhances reasoning logicality and interpretability. On multiple standard MQA benchmarks, ORACLE achieves accuracy competitive with state-of-the-art models such as DeepSeek-R1, while generating reasoning paths that exhibit superior consistency and verifiability.

Technology Category

• Knowledge Representation and Reasoning: Ontologies
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Question Answering

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Large Language Models (LLMs), despite their success in question answering, exhibit limitations in complex multi-hop question answering (MQA) tasks that necessitate non-linear, structured reasoning. This limitation stems from their inability to adequately capture deep conceptual relationships between entities. To overcome this challenge, we present **ORACLE** (**O**ntology-driven **R**easoning **A**nd **C**hain for **L**ogical **E**ucidation), a training-free framework that combines LLMs' generative capabilities with the structural benefits of knowledge graphs. Our approach operates through three stages: (1) dynamic construction of question-specific knowledge ontologies using LLMs, (2) transformation of these ontologies into First-Order Logic reasoning chains, and (3) systematic decomposition of the original query into logically coherent sub-questions. Experimental results on several standard MQA benchmarks show that our framework achieves highly competitive performance, rivaling current state-of-the-art models like DeepSeek-R1. Detailed analyses further confirm the effectiveness of each component, while demonstrating that our method generates more logical and interpretable reasoning chains than existing approaches.

Problem

• Enhance multi-hop question answering in LLMs
• Capture deep conceptual entity relationships
• Generate logical and interpretable reasoning chains

Innovation

• Dynamic construction of question-specific knowledge ontologies
• Transformation of ontologies into First-Order Logic chains
• Systematic decomposition into logically coherent sub-questions