Ensemble Threshold Calibration for Stable Sensitivity Control

2025-10-02

AI Summary

Large-scale spatial and entity matching suffers from imprecise recall control (bias often exceeding several percentage points), high variance, and substantial manual verification costs. Method: This paper proposes an end-to-end calibration framework integrating a multi-threshold estimator, decile-based score stratification, inverse-variance-weighted subsample ensembling, alongside equi-grid bounding-box filtering, CSR-based candidate representation, deterministic xxHash bootstrapping, and a lightweight neural ranker. Contribution/Results: It achieves, for the first time under highly skewed score distributions, sub-percentage-point (<1%) variance in recall control while precisely attaining target recall rates. Evaluated on real cadastral datasets of 6.31M and 67.34M pairs, the method significantly reduces redundant verification, enables full pipeline deployment on a single TPU v3 core, and ensures reproducibility, low variance, and hardware efficiency.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Search and Optimization: Distributed Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Querying, indexing, and retrieval in Web-related graphs

Abstract

Precise recall control is critical in large-scale spatial conflation and entity-matching tasks, where missing even a few true matches can break downstream analytics, while excessive manual review inflates cost. Classical confidence-interval cuts such as Clopper-Pearson or Wilson provide lower bounds on recall, but they routinely overshoot the target by several percentage points and exhibit high run-to-run variance under skewed score distributions. We present an end-to-end framework that achieves exact recall with sub-percent variance over tens of millions of geometry pairs, while remaining TPU-friendly. Our pipeline starts with an equigrid bounding-box filter and compressed sparse row (CSR) candidate representation, reducing pair enumeration by two orders of magnitude. A deterministic xxHash bootstrap sample trains a lightweight neural ranker; its scores are propagated to all remaining pairs via a single forward pass and used to construct a reproducible, score-decile-stratified calibration set. Four complementary threshold estimators - Clopper-Pearson, Jeffreys, Wilson, and an exact quantile - are aggregated via inverse-variance weighting, then fused across nine independent subsamples. This ensemble reduces threshold variance compared to any single method. Evaluated on two real cadastral datasets (approximately 6.31M and 67.34M pairs), our approach consistently hits a recall target within a small error, decreases redundant verifications relative to other calibrations, and runs end-to-end on a single TPU v3 core.

Problem

• Achieving precise recall control in large-scale entity matching tasks
• Reducing threshold variance and overshoot in confidence-based methods
• Enabling stable sensitivity control with minimal manual verification cost

Innovation

• Ensemble threshold calibration for stable sensitivity control
• Equigrid bounding-box filter reduces pair enumeration
• Lightweight neural ranker with score-decile-stratified calibration