Credible Intervals for Knowledge Graph Accuracy Estimation

2025-02-26

AI Summary

Existing knowledge graph (KG) accuracy assessments predominantly rely on frequentist confidence intervals (CIs), which often lead to statistical misinterpretation and unreliable inference. To address this, we propose a Bayesian credible interval (CrI) framework—first systematically introduced for KG accuracy evaluation—grounded in rigorous posterior inference. We theoretically establish that CrIs yield superior statistical properties and enhanced interpretability for posterior accuracy estimation compared to CIs. To ensure both computational efficiency and robustness at scale, we design an adaptive highest posterior density (aHPD) algorithm, enabling rapid and reliable interval estimation on large KGs. Extensive experiments on real-world KGs demonstrate that our approach significantly improves the reliability of accuracy estimates, strengthens statistical guarantees (e.g., calibrated coverage), and achieves higher computational efficiency. This work establishes a novel, principled paradigm for KG quality assessment grounded in Bayesian inference.

Technology Category

• Reasoning under Uncertainty: Probabilistic Inference
• Machine Learning: Calibration & Uncertainty Quantification
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Graph Algorithms and Modeling for the Web: Querying, indexing, and retrieval in Web-related graphs
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Knowledge Graphs (KGs) are widely used in data-driven applications and downstream tasks, such as virtual assistants, recommendation systems, and semantic search. The accuracy of KGs directly impacts the reliability of the inferred knowledge and outcomes. Therefore, assessing the accuracy of a KG is essential for ensuring the quality of facts used in these tasks. However, the large size of real-world KGs makes manual triple-by-triple annotation impractical, thereby requiring sampling strategies to provide accuracy estimates with statistical guarantees. The current state-of-the-art approaches rely on Confidence Intervals (CIs), derived from frequentist statistics. While efficient, CIs have notable limitations and can lead to interpretation fallacies. In this paper, we propose to overcome the limitations of CIs by using emph{Credible Intervals} (CrIs), which are grounded in Bayesian statistics. These intervals are more suitable for reliable post-data inference, particularly in KG accuracy evaluation. We prove that CrIs offer greater reliability and stronger guarantees than frequentist approaches in this context. Additionally, we introduce emph{a}HPD, an adaptive algorithm that is more efficient for real-world KGs and statistically robust, addressing the interpretive challenges of CIs.

Problem

• Estimating Knowledge Graph accuracy efficiently
• Overcoming Confidence Intervals' limitations
• Introducing Bayesian-based Credible Intervals

Innovation

• Bayesian Credible Intervals
• Adaptive HPD algorithm
• Knowledge Graph accuracy