Scalable Join Inference for Large Context Graphs

2026-03-04

AI Summary

This work addresses the challenge of inferring valid inter-entity connections from structured databases when constructing context graphs, a process often hindered by spurious links that introduce ambiguity and redundancy. To mitigate this, the authors propose a two-stage hybrid approach that integrates statistical pruning with large language model (LLM) reasoning: first identifying primary key candidates and then detecting inclusion dependencies, while dynamically refining connection inference using query history. By synergistically combining data-driven statistical analysis with LLM-based semantic understanding, the method significantly reduces hallucinations and false positives without compromising scalability. Experimental evaluation on TPC-DS, TPC-H, BIRD-Dev, and real-world production workloads demonstrates that the approach achieves high-precision connection inference—ranging from 78% to 100%—on well-structured schemas.

Technology Category

• Data Mining & Knowledge Management: Linked Open Data, Knowledge Graphs & KB Completion
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Context graphs are essential for modern AI applications including question answering, pattern discovery, and data analysis. Building accurate context graphs from structured databases requires inferring join relationships between entities. Invalid joins introduce ambiguity and duplicate records, compromising graph quality. We present a scalable join inference approach combining statistical pruning with Large Language Model (LLM) reasoning. Unlike purely statistics-based methods, our hybrid approach mimics human semantic understanding while mitigating LLM hallucination through data-driven inference. We first identify primary key candidates and use LLMs for adjudication, then detect inclusion dependencies with the same two-stage process. This statistics-LLM combination scales to large schemas while maintaining accuracy and minimizing false positives. We further leverage the database query history to refine the join inferences over time as the query workloads evolve. Our evaluation on TPC-DS, TPC-H, BIRD-Dev, and production workloads demonstrates that the approach achieves high precision (78-100%) on well-structured schemas, while highlighting the inherent difficulty of join discovery in poorly normalized settings.

Problem

• join inference
• context graphs
• entity relationships
• schema integration
• data quality

Innovation

• join inference
• large language models
• statistical pruning
• context graphs
• schema understanding