Accurate Evaluation of Quickest Changepoint Detectors via Non-parametric Survival Analysis

2026-05-11

AI Summary

This study addresses the challenge of evaluating change-point detectors under limited and irregularly sampled observation sequences, where conventional metrics such as Average Run Length (ARL) and Average Detection Delay (ADD) fail to provide accurate performance assessments. To overcome this limitation, the authors introduce survival analysis into this evaluation task for the first time, proposing nonparametric metrics—KM-ARL and KM-ADD—based on the Kaplan-Meier estimator. These metrics effectively model detection probabilities in truncated sequences without requiring distributional assumptions and possess asymptotic unbiasedness. Theoretical analysis and extensive experiments on both synthetic and real-world data demonstrate the robustness and interpretability of the proposed approach. An accompanying Python implementation has been made publicly available.

Abstract

We propose non-parametric estimators for the average run length (ARL) and average detection delay (ADD) in quickest changepoint detection (QCD) under finite and irregular sequence lengths. Although ARL and ADD are widely used as optimality criteria in theoretical and simulation studies, their application to real-world datasets is hindered by limited and irregular sequence lengths. To address this issue, we propose non-parametric estimators for the ARL and ADD, termed KM-ARL and KM-ADD, by drawing an analogy between QCD and survival analysis to model detection probabilities under sequence truncation. We derive estimation bias bounds and prove that they are asymptotically unbiased unless extrapolation is required. Experiments on simulated and real-world datasets demonstrate their practical utility, enhancing robustness against limited and irregular sequence lengths, improving interpretability, and facilitating empirical, intuitive model selection. Our Python code is provided at https://github.com/TaikiMiyagawa/Kaplan-Meier-Average-Run-Length, offering ready-to-use implementations for practitioners.

Problem

• quickest changepoint detection
• average run length
• average detection delay
• irregular sequence lengths
• non-parametric estimation

Innovation

• quickest changepoint detection
• non-parametric estimation
• survival analysis
• average run length
• Kaplan-Meier estimator