Measuring Graph-to-Graph Semantic Similarity in Knowledge Graphs: An Empirical Evaluation of Knowledge Graph Embeddings

2026-06-27

AI Summary

This study addresses the limitations of existing knowledge graph embedding methods, which primarily model at the triple level and struggle to capture graph-level semantic similarity, while traditional structural alignment approaches often lack semantic consistency. The work presents the first systematic evaluation of knowledge graph embeddings for graph-to-graph semantic matching tasks, introducing a novel dataset constructed via document rewriting to reflect semantic similarity. Two lightweight and efficient embedding-based scoring functions are proposed: Maximum Pairwise Entity Similarity (EmbPairSim) and Frequency-Weighted Centroid Similarity (AvgEmbSim). Experimental results on WikiText-2 and CC-News demonstrate that EmbPairSim achieves up to a 5.3-point MRR improvement over Sentence-BERT, confirming the effectiveness of knowledge graph embeddings as compact yet informative signals for graph-level semantic matching.

Abstract

A Knowledge Graph (KG) represents facts as structured triples and is widely used to organize relational knowledge across diverse domains. Just as textual information ranges from words and sentences to complete documents, KG information can be interpreted at multiple levels, from entities, relations, and triples to subgraphs and entire KGs. However, existing KG embedding methods mainly focus on entities, relations, and triples, leaving graph-level semantics largely unaddressed. Conventional graph-level methods, which typically compare graphs based on structural patterns, are also insufficient because structural similarity alone cannot guarantee semantic similarity between KGs. To evaluate how well different methods capture such graph-level semantic information, we study graph-to-graph semantic similarity, which determines whether a pair of KGs represents semantically corresponding underlying information. To obtain reliable ground-truth correspondences, we construct a semantic matching dataset by modifying text documents, extracting KGs from both original and modified documents, and transferring their known correspondences to KG pairs. We compare text-based, structure-based, and KG embedding-based approaches on each dataset. For the KG embedding-based approach, we introduce two scoring functions: \textit{EmbPairSim}, which uses maximal pairwise entity similarity, and \textit{AvgEmbSim}, which uses a frequency-weighted centroid. Experiments on WikiText-2 and CC-News show that \textit{EmbPairSim} achieves up to 5.3 pp higher MRR than Sentence-BERT while using substantially fewer parameters. These results suggest that KGE representations can serve as compact and effective signals for graph-to-graph semantic similarity in KGs. Our code is available at https://github.com/SeungRyeolBaek/KG-to-KG-Semantic-Similarity.

Problem

• Knowledge Graph
• Semantic Similarity
• Graph Embedding
• Graph-to-Graph Similarity
• Knowledge Representation

Innovation

• graph-to-graph semantic similarity
• knowledge graph embeddings
• EmbPairSim
• semantic matching dataset
• graph-level semantics