DUST: A Duality-Based Pruning Method For Exact Multiple Change-Point Detection

2025-07-03

AI Summary

Detecting multiple change points in large-scale time series under sparse, multi-parameter settings remains challenging; existing methods (e.g., PELT, FPOP) suffer from low computational efficiency or poor model adaptability. Method: We propose DUST—a framework based on penalized likelihood optimization under exponential-family distributions—that introduces strong duality to the non-convex pruning problem for the first time. Leveraging a dual-function thresholding mechanism, DUST enables efficient candidate change-point pruning within dynamic programming. Contribution/Results: DUST achieves both high accuracy and scalability, supports arbitrary-dimensional parameter spaces and non-Gaussian models, and unifies the theoretical elegance of PELT with the computational efficiency of FPOP. Experiments demonstrate that DUST significantly outperforms state-of-the-art methods across diverse change-point patterns and statistical models. It successfully identifies variance change points in mouse monitoring data, accurately recovering the optimal structural segmentation.

Technology Category

• Constraint Satisfaction and Optimization: Distributed CSP/Optimization
• Reasoning under Uncertainty: Stochastic Optimization
• Search and Optimization: Distributed Search

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization

Abstract

We tackle the challenge of detecting multiple change points in large time series by optimising a penalised likelihood derived from exponential family models. While dynamic programming algorithms can solve this task exactly with at most quadratic time complexity, recent years have seen the development of pruning strategies to improve their time efficiency. However, the two existing approaches have notable limitations: PELT struggles with pruning efficiency in sparse-change scenarios, while FPOP's structure is ill-suited for multi-parametric settings. To address these issues, we introduce the DUal Simple Test (DUST) framework, which prunes candidates by evaluating a dual function against a threshold. This approach is highly flexible and broadly applicable to parametric models of any dimension. Under mild assumptions, we establish strong duality for the underlying non-convex pruning problem. We demonstrate DUST's effectiveness across various change point regimes and models. In particular, for one-parametric models, DUST matches the simplicity of PELT with the efficiency of FPOP. Its use is especially advantageous for non-Gaussian models. Its use is especially advantageous for non-Gaussian models. Finally, we apply DUST to mouse monitoring time series under a change-in-variance model, illustrating its ability to recover the optimal change point structure efficiently.

Problem

• Detect multiple change points in large time series efficiently
• Overcome pruning limitations in sparse and multi-parametric scenarios
• Provide flexible framework for non-Gaussian and high-dimensional models

Innovation

• DUST framework evaluates dual function for pruning
• Flexible for parametric models of any dimension
• Strong duality for non-convex pruning problem