Entity Resolution via Batched Oracle Queries

2026-06-23

AI Summary

This work addresses the challenge of entity resolution under strict batch query constraints, where dataset sizes far exceed per-query batch limits and individual batches cannot guarantee inclusion of all records pertaining to the same entity. The paper formally defines this batch-constrained entity resolution problem for the first time, proves that optimal batch selection is NP-hard, and proposes an efficient algorithm under natural assumptions on entity size distributions. By integrating combinatorial optimization with clustering theory, the method devises an adaptive querying strategy that dynamically constructs batches using prior knowledge of entity sizes, thereby controlling cost on a pay-as-you-go basis while maximizing recall at each step. Experiments on six real-world datasets demonstrate that the approach significantly outperforms state-of-the-art baselines, achieving higher recall under stringent query budget constraints.

Abstract

We consider an oracle that processes a limited batch of records at a time and clusters those that refer to the same real-world entity. We study how to interrogate such an oracle to resolve entities in a dataset whose size is far larger than a single batch, and where no batch is guaranteed to contain all records of any given entity. We aim at a pay-as-you-go approach, to have full control over the costs (the number of oracle consults), while achieving the highest possible recall at every step. We formally cast this problem as batched entity resolution, prove that selecting optimal batches is NP-hard, and provide an optimal solution under a natural condition on entity sizes. Finally, we evaluate our approach on six datasets and show its superiority over state-of-the-art baselines.

Problem

• Entity Resolution
• Batched Oracle Queries
• Pay-as-you-go
• NP-hard
• Recall

Innovation

• batched entity resolution
• oracle queries
• pay-as-you-go
• NP-hard optimization
• entity clustering