Data Leakage Inflates Generalizability of Power Outage Prediction Models

2026-08-25

AI Summary

研究通过对比不同方法论选择对美国东海岸2018-2023年停电预测模型性能的影响，揭示了现有模型在新条件下的泛化能力受限，并提出改进数据覆盖和评估协议的建议。

Abstract

Power outage prediction models are increasingly used in assessments of climate-driven infrastructure risk, yet current evaluation practices obscure whether these models generalize to the novel conditions such applications require. We identify three common methodological choices in power outage prediction models that influence their ability to generalize across spatial, temporal, and event-based settings. We compare the predictive performance impacts of different methodological decisions using publicly available data for the U.S. East Coast from 2018 to 2023 and feature sets derived from weather reanalysis and land-cover data, and embeddings from a GeoAI foundation model (Prithvi WxC). Specifically, we assess model performance under multiple test selection strategies, including unfiltered random splits, leave-one-state-out, and leave-one-event-out designs, which increasingly approximate real-world deployment conditions. While random train-test splits yield strong performance, we show that these results are inflated by spatial and temporal autocorrelation. Under spatial and temporal holdout experiments, predictive accuracy degrades substantially, with models often failing to outperform a simple null baseline. Incorporating GeoAI foundation model embeddings yields limited and inconsistent improvements, primarily for spatial generalization, and does not resolve poor event-level transferability. These findings suggest that, given current data availability and evaluation practices, publicly trained outage prediction models offer limited and uncertain operational value. Progress will likely require improved data coverage, more realistic evaluation protocols, and a shift in focus from marginal modeling advances toward addressing structural data constraints.

Problem

• Power Outage Prediction
• Generalizability
• Climate-Driven Infrastructure Risk
• Spatial and Temporal Autocorrelation
• Evaluation Practices

Innovation

• spatial and temporal autocorrelation
• generalization
• GeoAI foundation model embeddings
• data coverage
• evaluation protocols