PatchAD: A Lightweight Patch-based MLP-Mixer for Time Series Anomaly Detection

2024-01-18

AI Summary

Time-series anomaly detection under label scarcity suffers from weak representation capability and excessive parameter redundancy in existing reconstruction-based methods. Method: We propose a lightweight multi-scale block-wise MLP-Mixer architecture, featuring a novel four-branch collaborative structure and a dual-projection constraint module. Our approach integrates patch embedding, multi-scale feature modeling, contrastive learning, and end-to-end optimization—achieving a compact 3.2 MB parameter footprint while preserving model capacity and representation robustness to mitigate representation degradation. Contribution/Results: Evaluated on nine cross-domain benchmark datasets, our method consistently outperforms 30 state-of-the-art baselines, achieving average improvements of 50.5% in F1-score, 7.8% in Adjusted F1 (Aff-F1), and 10.0% in AUC—demonstrating superior efficiency, generalizability, and robustness under limited supervision.

Technology Category

• Machine Learning: Time-Series/Data Streams
• Computer Vision: Multi-modal Vision
• Data Mining & Knowledge Management: Anomaly/Outlier Detection

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Anomaly detection in time series analysis is a pivotal task, yet it poses the challenge of discerning normal and abnormal patterns in label-deficient scenarios. While prior studies have largely employed reconstruction-based approaches, which limits the models' representational capacities. Moreover, existing deep learning-based methods are not sufficiently lightweight. Addressing these issues, we present PatchAD, our novel, highly efficient multiscale patch-based MLP-Mixer architecture that utilizes contrastive learning for representation extraction and anomaly detection. With its four distinct MLP Mixers and innovative dual project constraint module, PatchAD mitigates potential model degradation and offers a lightweight solution, requiring only $3.2$MB. Its efficacy is demonstrated by state-of-the-art results across $9$ datasets sourced from different application scenarios, outperforming over $30$ comparative algorithms. PatchAD significantly improves the classical F1 score by $50.5%$, the Aff-F1 score by $7.8%$, and the AUC by $10.0%$. The code is publicly available. url{https://github.com/EmorZz1G/PatchAD}

Problem

• Detect anomalies in label-deficient time series data
• Overcome limitations of reconstruction-based representation learning
• Provide lightweight deep learning solution for anomaly detection

Innovation

• Multiscale patch-based MLP-Mixer architecture
• Contrastive learning for anomaly detection
• Lightweight with only 0.403M parameters