Strong Linear Baselines Strike Back: Closed-Form Linear Models as Gaussian Process Conditional Density Estimators for TSAD

2026-01-31

AI Summary

This work challenges the prevailing reliance on computationally expensive deep learning models for time series anomaly detection by demonstrating the untapped potential of linear methods. The authors propose a closed-form linear autoregressive anomaly scoring approach based on ordinary least squares (OLS) and theoretically establish its equivalence to conditional density estimation under a Gaussian process with finite historical context. This formulation enables effective detection of diverse anomaly types. Extensive experiments show that the method achieves accuracy on par with or superior to state-of-the-art deep models across multiple univariate and multivariate benchmarks, while offering inference speeds several orders of magnitude faster and drastically reduced computational overhead—thereby questioning the necessity of complex deep architectures for this task.

Technology Category

• Machine Learning: Time-Series/Data Streams
• Data Mining & Knowledge Management: Anomaly/Outlier Detection
• Search and Optimization: Learning to Search

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Research in time series anomaly detection (TSAD) has largely focused on developing increasingly sophisticated, hard-to-train, and expensive-to-infer neural architectures. We revisit this paradigm and show that a simple linear autoregressive anomaly score with the closed-form solution provided by ordinary least squares (OLS) regression consistently matches or outperforms state-of-the-art deep detectors. From a theoretical perspective, we show that linear models capture a broad class of anomaly types, estimating a finite-history Gaussian process conditional density. From a practical side, across extensive univariate and multivariate benchmarks, the proposed approach achieves superior accuracy while requiring orders of magnitude fewer computational resources. Thus, future research should consistently include strong linear baselines and, more importantly, develop new benchmarks with richer temporal structures pinpointing the advantages of deep learning models.

Problem

• time series anomaly detection
• linear models
• deep learning baselines
• computational efficiency
• Gaussian process

Innovation

• linear autoregressive model
• ordinary least squares
• Gaussian process
• time series anomaly detection
• closed-form solution