Asymptotically Optimal Change Detection for Unnormalized Pre- and Post-Change Distributions

2024-10-18

arXiv.org

AI Summary

This paper addresses change-point detection for unnormalized distributions—such as those arising in ferromagnetic and thermodynamic physical systems—where the intractable normalization constant precludes direct application of classical CUSUM statistics. To overcome this challenge, we propose LPA-CUSUM: the first method leveraging thermodynamic integration to construct an unbiased estimator of the log-partition ratio, thereby enabling an asymptotically unbiased estimation of the CUSUM statistic. We theoretically establish that LPA-CUSUM achieves asymptotically optimal detection delay and derive a quantitative relationship between sample size and delay. Numerical experiments demonstrate that LPA-CUSUM significantly outperforms existing baseline methods in terms of detection delay, and we provide principled guidelines for tuning its adjustable parameters.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Web Mining and Content Analysis: Models for Web evolution
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

This paper addresses the problem of detecting changes when only unnormalized pre- and post-change distributions are accessible. This situation happens in many scenarios in physics such as in ferromagnetism, crystallography, magneto-hydrodynamics, and thermodynamics, where the energy models are difficult to normalize. Our approach is based on the estimation of the Cumulative Sum (CUSUM) statistics, which is known to produce optimal performance. We first present an intuitively appealing approximation method. Unfortunately, this produces a biased estimator of the CUSUM statistics and may cause performance degradation. We then propose the Log-Partition Approximation Cumulative Sum (LPA-CUSUM) algorithm based on thermodynamic integration (TI) in order to estimate the log-ratio of normalizing constants of pre- and post-change distributions. It is proved that this approach gives an unbiased estimate of the log-partition function and the CUSUM statistics, and leads to an asymptotically optimal performance. Moreover, we derive a relationship between the required sample size for thermodynamic integration and the desired detection delay performance, offering guidelines for practical parameter selection. Numerical studies are provided demonstrating the efficacy of our approach.

Problem

• Detecting changes in unnormalized distributions
• Optimal performance in change detection
• Unbiased estimation using LPA-CUSUM algorithm

Innovation

• Unnormalized distribution change detection
• LPA-CUSUM algorithm implementation
• Thermodynamic integration for estimation