Low Rank Transformer for Multivariate Time Series Anomaly Detection and Localization

2026-02-09

AI Summary

This work addresses the lack of theoretical foundations in multivariate time series anomaly detection and variable-level localization, where existing methods often fail to precisely identify the root causes of anomalies. By revealing an intrinsic connection between Transformers and statistical time series models, the authors propose ALoRa-T and ALoRa-Loc—introducing low-rank regularization into the self-attention mechanism for the first time to construct an interpretable low-rank self-attention architecture. They further design an Attention Low-Rank score and a variable-association-based localization strategy. Extensive experiments on multiple real-world datasets demonstrate that the proposed approach significantly outperforms state-of-the-art methods in both anomaly detection and variable-level localization, achieving a compelling balance between theoretical rigor and practical performance.

Technology Category

• Machine Learning: Time-Series/Data Streams
• Data Mining & Knowledge Management: Anomaly/Outlier Detection
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Multivariate time series (MTS) anomaly diagnosis, which encompasses both anomaly detection and localization, is critical for the safety and reliability of complex, large-scale real-world systems. The vast majority of existing anomaly diagnosis methods offer limited theoretical insights, especially for anomaly localization, which is a vital but largely unexplored area. The aim of this contribution is to study the learning process of a Transformer when applied to MTS by revealing connections to statistical time series methods. Based on these theoretical insights, we propose the Attention Low-Rank Transformer (ALoRa-T) model, which applies low-rank regularization to self-attention, and we introduce the Attention Low-Rank score, effectively capturing the temporal characteristics of anomalies. Finally, to enable anomaly localization, we propose the ALoRa-Loc method, a novel approach that associates anomalies to specific variables by quantifying interrelationships among time series. Extensive experiments and real data analysis, show that the proposed methodology significantly outperforms state-of-the-art methods in both detection and localization tasks.

Problem

• Multivariate Time Series
• Anomaly Detection
• Anomaly Localization
• Transformer
• Low-Rank

Innovation

• Low-Rank Attention
• Transformer
• Multivariate Time Series
• Anomaly Localization
• Anomaly Detection