Pi-Transformer: A Physics-informed Attention Mechanism for Time Series Anomaly Detection

2025-09-24

AI Summary

Multivariate time-series anomalies often arise from temporal misalignment and inter-channel phase desynchronization—not isolated outliers. To address this, we propose a physics-informed dual-path attention mechanism: one path models scale-wise self-similarity to capture temporal invariance, while the other encodes phase synchrony to characterize dynamic coordination; their joint output calibrates reconstruction error. We further design a detection scoring function that fuses alignment-weighted energy and mismatched signal representations. Built upon a Transformer architecture, the model is trained end-to-end using a composite objective comprising reconstruction loss and divergence regularization; prior-guided attention is constrained by smoothness and lightweight distillation of data-driven statistical features. Evaluated on five benchmarks—SMD, MSL, SMAP, SWaT, and PSM—the method achieves state-of-the-art or near-state-of-the-art F1 scores, with particularly strong gains in detecting temporal discontinuities and phase anomalies.

Technology Category

• Machine Learning: Time-Series/Data Streams
• Data Mining & Knowledge Management: Anomaly/Outlier Detection
• Computer Vision: Multi-modal Vision

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs
• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Anomalies in multivariate time series often arise from temporal context and cross-channel coordination rather than isolated outliers. We present Pi-Transformer, a physics-informed transformer with two attention pathways: a data-driven series attention and a smoothly evolving prior attention that encodes temporal invariants such as scale-related self-similarity and phase synchrony. The prior acts as a stable reference that calibrates reconstruction error. During training, we pair a reconstruction objective with a divergence term that encourages agreement between the two attentions while keeping them meaningfully distinct; the prior is regularised to evolve smoothly and is lightly distilled towards dataset-level statistics. At inference, the model combines an alignment-weighted reconstruction signal (Energy) with a mismatch signal that highlights timing and phase disruptions, and fuses them into a single score for detection. Across five benchmarks (SMD, MSL, SMAP, SWaT, and PSM), Pi-Transformer achieves state-of-the-art or highly competitive F1, with particular strength on timing and phase-breaking anomalies. Case analyses show complementary behaviour of the two streams and interpretable detections around regime changes. Embedding physics-informed priors into attention yields a calibrated and robust approach to anomaly detection in complex multivariate systems. Code is publicly available at this GitHub repositoryfootnote{https://github.com/sepehr-m/Pi-Transformer}.

Problem

• Detecting anomalies in multivariate time series using physics-informed attention
• Addressing temporal context and cross-channel coordination anomalies
• Combining data-driven and prior attention for robust detection

Innovation

• Physics-informed transformer with dual attention pathways
• Prior attention encodes temporal invariants like self-similarity
• Combines alignment-weighted reconstruction with mismatch signals