Surprised by Attention: Predictable Query Dynamics for Time Series Anomaly Detection

2026-03-13

AI Summary

This work proposes AxonAD, an unsupervised anomaly detection method for multivariate time series that addresses the limitation of traditional reconstruction-based approaches, which often fail to capture anomalies arising from abrupt shifts in cross-channel dependency structures rather than mere amplitude deviations. AxonAD uniquely models query vectors in multi-head attention as short-term predictable signals, forecasting future queries from historical context and jointly leveraging reconstruction error and query-mismatch scores for anomaly discrimination. The method incorporates a history-only query predictor and an EMA-based target encoder to integrate both structural and amplitude information, enhancing sensitivity to subtle anomalies. Evaluated on in-vehicle telemetry data and the TSB-AD benchmark (17 datasets, 180 sequences), AxonAD significantly outperforms strong baselines, achieving superior anomaly ranking and temporal localization accuracy without requiring threshold tuning or range-aware metrics.

Technology Category

• Data Mining & Knowledge Management: Anomaly/Outlier Detection
• Machine Learning: Time-Series/Data Streams
• Multiagent Systems: Adversarial Agents

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Multivariate time series anomalies often manifest as shifts in cross-channel dependencies rather than simple amplitude excursions. In autonomous driving, for instance, a steering command might be internally consistent but decouple from the resulting lateral acceleration. Residual-based detectors can miss such anomalies when flexible sequence models still reconstruct signals plausibly despite altered coordination. We introduce AxonAD, an unsupervised detector that treats multi-head attention query evolution as a short horizon predictable process. A gradient-updated reconstruction pathway is coupled with a history-only predictor that forecasts future query vectors from past context. This is trained via a masked predictor-target objective against an exponential moving average (EMA) target encoder. At inference, reconstruction error is combined with a tail-aggregated query mismatch score, which measures cosine deviation between predicted and target queries on recent timesteps. This dual approach provides sensitivity to structural dependency shifts while retaining amplitude-level detection. On proprietary in-vehicle telemetry with interval annotations and on the TSB-AD multi-variate suite (17 datasets, 180 series) with threshold-free and range-aware metrics, AxonAD improves ranking quality and temporal localization over strong baselines. Ablations confirm that query prediction and combined scoring are the primary drivers of the observed gains. Code is available at the URL https://github.com/iis-esslingen/AxonAD.

Problem

• multivariate time series
• anomaly detection
• cross-channel dependencies
• structural dependency shifts
• unsupervised detection

Innovation

• attention query dynamics
• unsupervised anomaly detection
• multivariate time series
• predictable query evolution
• cosine deviation scoring