ARIMA_PLUS: Large-scale, Accurate, Automatic and Interpretable In-Database Time Series Forecasting and Anomaly Detection in Google BigQuery

2025-10-28

AI Summary

Addressing the challenge of jointly optimizing efficiency, accuracy, and interpretability in large-scale time series forecasting and anomaly detection, this paper introduces ARIMA_PLUS—a BigQuery-native framework enabling fully automated, SQL-driven modeling. Methodologically, it adopts a modular architecture that explicitly disentangles holiday effects, trend, seasonality, and anomalies, enhancing business interpretability while unifying forecasting and anomaly detection under a single framework. Leveraging an enhanced ARIMA structure and cloud-native distributed computing, ARIMA_PLUS delivers end-to-end automation for data cleaning, model selection, and training. Empirically, it outperforms classical statistical methods and state-of-the-art deep learning models across all 42 datasets in the Monash benchmark. Scalability is demonstrated by processing 100 million time series in just 1.5 hours (throughput >18,000 series/sec), confirming industrial-grade scalability and high automation.

Technology Category

• Machine Learning: Time-Series/Data Streams
• Data Mining & Knowledge Management: Anomaly/Outlier Detection
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Web Mining and Content Analysis: Large pretrained models with web data
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization

Abstract

Time series forecasting and anomaly detection are common tasks for practitioners in industries such as retail, manufacturing, advertising and energy. Two unique challenges stand out: (1) efficiently and accurately forecasting time series or detecting anomalies in large volumes automatically; and (2) ensuring interpretability of results to effectively incorporate business insights. We present ARIMA_PLUS, a novel framework to overcome these two challenges by a unique combination of (a) accurate and interpretable time series models and (b) scalable and fully managed system infrastructure. The model has a sequential and modular structure to handle different components of the time series, including holiday effects, seasonality, trend, and anomalies, which enables high interpretability of the results. Novel enhancements are made to each module, and a unified framework is established to address both forecasting and anomaly detection tasks simultaneously. In terms of accuracy, its comprehensive benchmark on the 42 public datasets in the Monash forecasting repository shows superior performance over not only well-established statistical alternatives (such as ETS, ARIMA, TBATS, Prophet) but also newer neural network models (such as DeepAR, N-BEATS, PatchTST, TimeMixer). In terms of infrastructure, it is directly built into the query engine of BigQuery in Google Cloud. It uses a simple SQL interface and automates tedious technicalities such as data cleaning and model selection. It automatically scales with managed cloud computational and storage resources, making it possible to forecast 100 million time series using only 1.5 hours with a throughput of more than 18000 time series per second. In terms of interpretability, we present several case studies to demonstrate time series insights it generates and customizability it offers.

Problem

• Automating large-scale time series forecasting and anomaly detection
• Ensuring interpretability of results for business insights
• Providing scalable infrastructure for efficient data processing

Innovation

• In-database scalable system for automatic time series forecasting
• Interpretable modular model handling trend seasonality anomalies
• SQL interface automating data cleaning and model selection