Interpretable Question Answering with Knowledge Graphs

2025-10-21

AI Summary

This paper addresses the challenge of building explainable question-answering (QA) systems without relying on large language models (LLMs) or retrieval-augmented generation (RAG), focusing instead on pure knowledge graph (KG)-driven semantic retrieval and reasoning. Method: We propose a two-stage framework: (1) automatic construction of structured QA pairs from documents and domain-specific KGs; and (2) end-to-end graph retrieval, path re-ranking, and natural language answer generation via entity/relation embeddings, fuzzy graph matching, and a lightweight semantic rewriting model. Contribution/Results: Our key innovation is using a compact rewriting model to refine semantic representations of KG entity-relation edges, significantly enhancing both interpretability and answer accuracy. Evaluated on the CRAG benchmark with LLaMA-3.2 and GPT-3.5-Turbo as judge models, our approach achieves 71.9% and 54.4% accuracy, respectively—demonstrating the effectiveness and competitiveness of KG-only reasoning for explainable QA.

Technology Category

• Natural Language Processing: Question Answering
• Knowledge Representation and Reasoning: Knowledge Acquisition
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

This paper presents a question answering system that operates exclusively on a knowledge graph retrieval without relying on retrieval augmented generation (RAG) with large language models (LLMs). Instead, a small paraphraser model is used to paraphrase the entity relationship edges retrieved from querying the knowledge graph. The proposed pipeline is divided into two main stages. The first stage involves pre-processing a document to generate sets of question-answer (QA) pairs. The second stage converts these QAs into a knowledge graph from which graph-based retrieval is performed using embeddings and fuzzy techniques. The graph is queried, re-ranked, and paraphrased to generate a final answer. This work includes an evaluation using LLM-as-a-judge on the CRAG benchmark, which resulted in accuracies of 71.9% and 54.4% using LLAMA-3.2 and GPT-3.5-Turbo, respectively.

Problem

• Develops knowledge graph-based QA without LLM retrieval
• Converts document QAs into queryable knowledge graphs
• Uses graph retrieval and paraphrasing for interpretable answers

Innovation

• Knowledge graph retrieval replaces RAG with LLMs
• Small paraphraser model paraphrases entity relationship edges
• Graph-based retrieval uses embeddings and fuzzy techniques