EMERGE: A Benchmark for Updating Knowledge Graphs with Emerging Textual Knowledge

2025-07-04

AI Summary

Knowledge graphs (KGs) lack effective mechanisms for automatically adapting to knowledge evolution reflected in unstructured text. Method: This paper proposes a continual KG construction and evaluation framework for dynamic KG updating, introducing KG-Update—the first large-scale temporally aligned dataset—comprising 10 Wikidata snapshots (2019–2025), corresponding Wikipedia paragraphs, and human-verified KG edit operations (376K paragraphs, 1.25M edits). Leveraging text–KG temporal alignment and edit operation identification, the framework enables traceable modeling from textual changes to KG updates. Contribution/Results: KG-Update bridges the gap between information extraction and knowledge evolution research, establishing the first reproducible, evaluable benchmark for continual KG learning and dynamic updating. It supports rigorous evaluation of temporal reasoning, edit prediction, and incremental KG construction, thereby advancing research on adaptive, time-aware knowledge representation.

Technology Category

• Data Mining & Knowledge Management: Linked Open Data, Knowledge Graphs & KB Completion
• Knowledge Representation and Reasoning: Knowledge Acquisition
• Machine Learning: Life-Long and Continual Learning

Application Category

• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Web Mining and Content Analysis: Models for Web evolution

Abstract

Knowledge Graphs (KGs) are structured knowledge repositories containing entities and relations between them. In this paper, we investigate the problem of automatically updating KGs over time with respect to the evolution of knowledge in unstructured textual sources. This problem requires identifying a wide range of update operations based on the state of an existing KG at a specific point in time. This contrasts with traditional information extraction pipelines, which extract knowledge from text independently of the current state of a KG. To address this challenge, we propose a method for lifelong construction of a dataset consisting of Wikidata KG snapshots over time and Wikipedia passages paired with the corresponding edit operations that they induce in a particular KG snapshot. The resulting dataset comprises 376K Wikipedia passages aligned with a total of 1.25M KG edits over 10 different snapshots of Wikidata from 2019 to 2025. Our experimental results highlight challenges in updating KG snapshots based on emerging textual knowledge, positioning the dataset as a valuable benchmark for future research. We will publicly release our dataset and model implementations.

Problem

• Automatically updating Knowledge Graphs with evolving textual knowledge
• Identifying diverse update operations based on KG state over time
• Creating a benchmark for lifelong KG updates from text

Innovation

• Lifelong dataset construction with Wikidata snapshots
• Aligns Wikipedia passages with KG edit operations
• Benchmark for updating KGs from emerging texts