Data Fusion for High-Resolution Estimation

2025-08-20

AI Summary

This study addresses the challenge of jointly leveraging low-resolution unbiased data (e.g., administrative statistics) and high-resolution biased data (e.g., online surveys) for fine-grained population health indicator estimation. We propose a sampling-bias-aware data fusion framework that models response probability as a function of observable covariates, assuming bias is captured by a linear function of sufficient statistics. Our method learns a joint distribution that minimizes KL divergence from the survey-derived distribution while satisfying aggregation constraints imposed by administrative data—thereby calibrating high-resolution estimates to an unbiased benchmark. Experiments across multiple spatiotemporal scales demonstrate significant improvements in accuracy and robustness over single-source baseline models. The framework establishes a novel paradigm for integrating heterogeneous observational data to enable granular, reliable public health monitoring.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Relational Probabilistic Models
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Web Mining and Content Analysis: Web data integration and cleaning
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

High-resolution estimates of population health indicators are critical for precision public health. We propose a method for high-resolution estimation that fuses distinct data sources: an unbiased, low-resolution data source (e.g. aggregated administrative data) and a potentially biased, high-resolution data source (e.g. individual-level online survey responses). We assume that the potentially biased, high-resolution data source is generated from the population under a model of sampling bias where observables can have arbitrary impact on the probability of response but the difference in the log probabilities of response between units with the same observables is linear in the difference between sufficient statistics of their observables and outcomes. Our data fusion method learns a distribution that is closest (in the sense of KL divergence) to the online survey distribution and consistent with the aggregated administrative data and our model of sampling bias. This method outperforms baselines that rely on either data source alone on a testbed that includes repeated measurements of three indicators measured by both the (online) Household Pulse Survey and ground-truth data sources at two geographic resolutions over the same time period.

Problem

• Fusing biased high-resolution and unbiased low-resolution data sources
• Estimating high-resolution population health indicators accurately
• Correcting sampling bias in online surveys using administrative data

Innovation

• Fuses unbiased low-resolution and biased high-resolution data
• Learns distribution minimizing KL divergence from survey data
• Outperforms single-source baselines on public health indicators