LinkQ: An LLM-Assisted Visual Interface for Knowledge Graph Question-Answering

2024-06-07

Visual ..

AI Summary

Knowledge Graph (KG) question answering requires proficiency in formal query languages (e.g., SPARQL, GraphQL), posing high entry barriers and increasing hallucination risks in large language models (LLMs). Method: We propose an LLM-driven visual QA framework that decomposes natural language questions into structured queries via multi-step parsing and iterative refinement. It introduces a novel closed-loop verification mechanism integrating LLMs with KG execution engines to strictly constrain answers to actual graph data. The framework further supports progressive clarification for open-ended questions and enables both targeted and exploratory analysis modes. Results: Qualitative evaluation with five KG practitioners confirmed the framework’s effectiveness. It significantly lowers the barrier for non-expert users to construct accurate queries, ensures result traceability and analytical flexibility, and substantially mitigates hallucination by grounding all outputs in verified KG facts.

Technology Category

• Natural Language Processing: Question Answering
• Data Mining & Knowledge Management: Linked Open Data, Knowledge Graphs & KB Completion
• Knowledge Representation and Reasoning: Knowledge Representation Languages

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web question answering

Abstract

We present LinkQ, a system that leverages a large language model (LLM) to facilitate knowledge graph (KG) query construction through natural language question-answering. Traditional approaches often require detailed knowledge of a graph querying language, limiting the ability for users – even experts – to acquire valuable insights from KGs. LinkQ simplifies this process by implementing a multistep protocol in which the LLM interprets a user’s question, then systematically converts it into a well-formed query. LinkQ helps users iteratively refine any open-ended questions into precise ones, supporting both targeted and exploratory analysis. Further, LinkQ guards against the LLM hallucinating outputs by ensuring users’ questions are only ever answered from ground truth KG data. We demonstrate the efficacy of LinkQ through a qualitative study with five KG practitioners. Our results indicate that practitioners find LinkQ effective for KG question-answering, and desire future LLM-assisted exploratory data analysis systems.

Problem

• Simplifying knowledge graph querying without requiring query language expertise
• Converting natural language questions into structured graph queries iteratively
• Preventing LLM hallucinations by grounding answers in KG data

Innovation

• LLM-assisted natural language query construction
• Multistep protocol for iterative question refinement
• Ground truth KG data prevents LLM hallucination