Towards Researcher Agents for Knowledge-Graph Question Answering

2026-08-07

AI Summary

Accurately translating natural language questions into SPARQL queries over knowledge graphs is challenged by lexical ambiguity, terminology misalignment, and semantic fidelity. This work proposes the first self-improving researcher agent framework that iteratively refines prompts, reasoning rules, and tool invocation logic through automated validation. Leveraging low-cost models to drive evolutionary improvements, the framework deploys large language models on DBpedia. After nine rounds of self-iteration, it achieves an overall accuracy of 0.22 on the 2025 DBpedia benchmark, revealing predicate selection as the primary bottleneck. The study further identifies misjudgments in the benchmark caused by attribute ambiguity, highlighting significant limitations in current evaluation protocols.

Abstract

Translating a natural-language question into a SPARQL query that can be executed against a large knowledge graph requires resolving lexical ambiguity, grounding surface terms in the target ontology, and producing graph patterns that are both syntactically valid and semantically faithful. We present an agentic text-to-SPARQL system that goes one step beyond static tool-using agents: a researcher agent that, after each round of inference on a validation set, proposes and tests changes to its own prompts, rules, and tool-orchestration code. We instantiate the loop on DBpedia, evolve nine successive versions of the agent driven by a low-cost reasoning model, and deploy the best-performing configuration with two stronger backbone models. The study yields three observations: (i) self-improvement converges quickly and then achieves 0.22 overall accuracy on the 2025 DBpedia validation set; (ii) the bottleneck is consistently in basic-graph-pattern predicate selection, not in SPARQL syntax or modifiers; and (iii) several benchmark items appear to penalise correct queries due to property ambiguity in DBpedia, suggesting that future Text-to-SPARQL benchmarks should be scored using a combination of machine translation and information retrieval metrics.

Problem

• Knowledge Graph
• Question Answering
• SPARQL
• Natural Language Processing
• Semantic Parsing

Innovation

• researcher agent
• self-improvement
• text-to-SPARQL
• knowledge graph question answering
• prompt evolution