All Relations Lead to Rome: Automated Knowledge Graph Creation and Question Generation

2026-06-21

AI Summary

Existing datasets struggle to simultaneously support vector retrieval and knowledge graph reasoning, often lacking structured knowledge aligned with textual corpora and high-quality factual question-answer pairs. To address this gap, this work proposes ARLtR, a unified framework that, for the first time, deeply integrates symbolic knowledge graphs with dense text representations to jointly learn embeddings of entities, relations, and text passages alongside corresponding question-answer pairs. Centered on Roman Empire history, the authors construct a new dataset comprising over 19,000 entities, 16,000 text chunks, and 8,400 rigorously curated factual QA pairs. The dataset has been publicly released on Hugging Face, offering a verifiable and evaluable benchmark for hybrid retrieval and semantics-guided approaches.

Abstract

Large language models have substantially improved information retrieval and question answering; however, existing datasets generally support either vector-based retrieval over unstructured text or reasoning over knowledge graphs, without providing a unified representation that combines both paradigms. Moreover, current benchmarks rarely provide ground-truth entities, relations, and fact-grounded question-answer pairs aligned with the underlying corpus. To address this gap, we introduce All Relations Lead to Rome (ARLtR), a unified framework for automated knowledge graph construction and fact-grounded question-answer generation. ARLtR jointly constructs a knowledge graph, embeddings, and question-answer pairs that are explicitly grounded in extracted entities, relations, and supporting textual evidence. We further instantiate the framework as a historical dataset centered on the Roman Empire, comprising over 19,000 entities, 16,000 chunks, and 8,400 question-answer pairs (https://huggingface.co/datasets/FaynePro/all-relations-lead-to-rome). By tightly coupling symbolic graph representations with dense retrieval representations, ARLtR facilitates the evaluation and development of hybrid retrieval systems and semantic steering approaches within a single coherent resource.

Problem

• knowledge graph
• question generation
• information retrieval
• fact-grounded QA
• unified representation

Innovation

• knowledge graph construction
• fact-grounded question generation
• hybrid retrieval
• dense retrieval
• symbolic representation