MSAD: A Deep Dive into Model Selection for Time series Anomaly Detection

2025-10-30

AI Summary

Time-series anomaly detection suffers from strong dependence of model performance on data characteristics and lacks automated model selection mechanisms. Method: This paper proposes the first AutoML-style model selection framework for anomaly detection, which learns a mapping from time-series features to optimal detectors using a classifier trained over 16 base models and 234 configurations. Evaluation across 1,980+ real-world time series demonstrates that the framework significantly outperforms any single detector while incurring inference overhead comparable to that of a single model. Contributions/Results: (1) It establishes and validates, for the first time, a time-series-feature-driven model selection paradigm; (2) it provides an extensible, plug-and-play AutoML baseline; and (3) it open-sources a unified benchmark and feature set, addressing a critical gap in systematic research on automated model selection for time-series anomaly detection.

Technology Category

• Machine Learning: Time-Series/Data Streams
• Data Mining & Knowledge Management: Anomaly/Outlier Detection
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Anomaly detection is a fundamental task for time series analytics with important implications for the downstream performance of many applications. Despite increasing academic interest and the large number of methods proposed in the literature, recent benchmarks and evaluation studies demonstrated that no overall best anomaly detection methods exist when applied to very heterogeneous time series datasets. Therefore, the only scalable and viable solution to solve anomaly detection over very different time series collected from diverse domains is to propose a model selection method that will select, based on time series characteristics, the best anomaly detection methods to run. Existing AutoML solutions are, unfortunately, not directly applicable to time series anomaly detection, and no evaluation of time series-based approaches for model selection exists. Towards that direction, this paper studies the performance of time series classification methods used as model selection for anomaly detection. In total, we evaluate 234 model configurations derived from 16 base classifiers across more than 1980 time series, and we propose the first extensive experimental evaluation of time series classification as model selection for anomaly detection. Our results demonstrate that model selection methods outperform every single anomaly detection method while being in the same order of magnitude regarding execution time. This evaluation is the first step to demonstrate the accuracy and efficiency of time series classification algorithms for anomaly detection, and represents a strong baseline that can then be used to guide the model selection step in general AutoML pipelines. Preprint version of an article accepted at the VLDB Journal.

Problem

• Selecting optimal anomaly detection methods for diverse time series datasets
• Evaluating time series classification approaches for model selection in anomaly detection
• Addressing lack of universal best anomaly detection method across heterogeneous datasets

Innovation

• Time series classification for model selection
• Evaluating 234 configurations across 1980 series
• Model selection outperforms single anomaly detection methods