Proxy-Guided Measurement Calibration

2026-03-10

AI Summary

This study addresses the inaccuracy in aggregated statistics arising from systematic measurement errors in survey and administrative records—such as biased county-level disaster loss reports—by proposing a causal graph–based calibration framework. The approach leverages proxy variables that depend solely on the true outcome and are independent of the bias mechanism, combined with a two-stage variational autoencoder to disentangle latent content variables from bias-inducing factors. This method achieves, for the first time, identifiable estimation of bias effects under systematic measurement error. Experimental evaluations on synthetic data, semi-synthetic benchmarks, and real-world disaster loss records demonstrate that the proposed framework substantially improves calibration accuracy.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Causality
• Computer Vision: Bias, Fairness & Privacy

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Security and Privacy: Large-scale security measurements

Abstract

Aggregate outcome variables collected through surveys and administrative records are often subject to systematic measurement error. For instance, in disaster loss databases, county-level losses reported may differ from the true damages due to variations in on-the-ground data collection capacity, reporting practices, and event characteristics. Such miscalibration complicates downstream analysis and decision-making. We study the problem of outcome miscalibration and propose a framework guided by proxy variables for estimating and correcting the systematic errors. We model the data-generating process using a causal graph that separates latent content variables driving the true outcome from the latent bias variables that induce systematic errors. The key insight is that proxy variables that depend on the true outcome but are independent of the bias mechanism provide identifying information for quantifying the bias. Leveraging this structure, we introduce a two-stage approach that utilizes variational autoencoders to disentangle content and bias latents, enabling us to estimate the effect of bias on the outcome of interest. We analyze the assumptions underlying our approach and evaluate it on synthetic data, semi-synthetic datasets derived from randomized trials, and a real-world case study of disaster loss reporting.

Problem

• measurement error
• outcome miscalibration
• systematic bias
• proxy variables
• calibration

Innovation

• proxy-guided calibration
• measurement error
• causal disentanglement
• variational autoencoder
• systematic bias