Benchmarking Counterfactual Prediction in Epidemic Time Series with Time-Varying Interventions

2026-06-04

AI Summary

Existing methods for causal inference in time series lack realistic benchmarks with ground-truth counterfactual outcomes, making it difficult to evaluate their performance under dynamic interventions. To address this gap, this work introduces a large-scale counterfactual forecasting benchmark for epidemiological dynamics, constructed using an agent-based model calibrated with real-world demographic, mobility, epidemiological, and policy data. The benchmark provides counterfactual trajectories for over 150 U.S. counties under diverse intervention scenarios—including static versus time-varying and single versus multiple policies—thereby capturing both realism and complex causal dynamics. This is the first benchmark to jointly satisfy these criteria, filling a critical void in existing datasets. Systematic evaluation of both established and state-of-the-art methods reveals substantial performance disparities in realistic dynamic intervention settings, highlighting key challenges in time series causal inference.

Abstract

Deep learning has enabled significant advances in time-series causal inference, yet progress remains constrained by the lack of realistic benchmarks with observable counterfactual outcomes. Existing datasets either rely on real-world observations without ground-truth counterfactuals or on simplified simulations that fail to capture complex causal dynamics. To address this gap, we develop a large-scale benchmark for counterfactual prediction in epidemic time series under dynamic interventions. Unlike existing benchmarks, it supports static and time-varying treatments, as well as both single-policy and multi-policy intervention settings, enabling evaluation of causal inference methods across a broad range of causal inference scenarios. Leveraging a calibrated agent-based model grounded in real-world demographic, mobility, epidemiological, and policy data, we generate realistic counterfactual trajectories across more than 150 U.S. counties. Using this benchmark, we evaluate widely used and state-of-the-art causal inference methods, revealing substantial performance differences and highlighting the challenges of realistic time-series causal reasoning.

Problem

• counterfactual prediction
• epidemic time series
• time-varying interventions
• causal inference
• benchmark

Innovation

• counterfactual prediction
• time-varying interventions
• epidemic time series
• agent-based simulation
• causal inference benchmark