Leveraging LLM-GNN Integration for Open-World Question Answering over Knowledge Graphs

2026-04-15

AI Summary

This work addresses the challenge of open-world question answering under incomplete or dynamically evolving knowledge graphs by proposing GLOW, a framework that transcends the conventional closed-world assumption. GLOW integrates pretrained graph neural networks (GNNs) with large language models (LLMs), leveraging the GNN to generate candidate answers and construct structured prompts that guide the LLM to jointly reason over symbolic and semantic information—enabling zero-shot question answering without retrieval or fine-tuning. Notably, GLOW achieves the first synergy between structure-aware reasoning and semantic understanding without relying on path assumptions or complete graph coverage. The authors also introduce GLOW-BENCH, the first benchmark specifically designed for open-world QA over incomplete knowledge graphs. Experiments demonstrate that GLOW substantially outperforms existing methods, achieving average gains of 38% and up to 53.3% on both standard benchmarks and GLOW-BENCH.

Technology Category

• Data Mining & Knowledge Management: Linked Open Data, Knowledge Graphs & KB Completion
• Natural Language Processing: Question Answering
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web question answering

Abstract

Open-world Question Answering (OW-QA) over knowledge graphs (KGs) aims to answer questions over incomplete or evolving KGs. Traditional KGQA assumes a closed world where answers must exist in the KG, limiting real-world applicability. In contrast, open-world QA requires inferring missing knowledge based on graph structure and context. Large language models (LLMs) excel at language understanding but lack structured reasoning. Graph neural networks (GNNs) model graph topology but struggle with semantic interpretation. Existing systems integrate LLMs with GNNs or graph retrievers. Some support open-world QA but rely on structural embeddings without semantic grounding. Most assume observed paths or complete graphs, making them unreliable under missing links or multi-hop reasoning. We present GLOW, a hybrid system that combines a pre-trained GNN and an LLM for open-world KGQA. The GNN predicts top-k candidate answers from the graph structure. These, along with relevant KG facts, are serialized into a structured prompt (e.g., triples and candidates) to guide the LLM's reasoning. This enables joint reasoning over symbolic and semantic signals, without relying on retrieval or fine-tuning. To evaluate generalization, we introduce GLOW-BENCH, a 1,000-question benchmark over incomplete KGs across diverse domains. GLOW outperforms existing LLM-GNN systems on standard benchmarks and GLOW-BENCH, achieving up to 53.3% and an average 38% improvement. GitHub code and data are available.

Problem

• Open-world Question Answering
• Knowledge Graphs
• Incomplete KGs
• Missing Knowledge Inference
• Multi-hop Reasoning

Innovation

• LLM-GNN integration
• open-world question answering
• structured prompting
• knowledge graph completion
• symbolic-semantic reasoning