This Time is Different: An Observability Perspective on Time Series Foundation Models

2025-05-20

AI Summary

This work addresses the modeling challenge of multivariate time-series forecasting for observability monitoring. We propose Toto—a foundational decoder-only architecture (151M parameters) specifically designed for observability data—and BOOM, the first large-scale benchmark comprising 350M real-world production observations across 2,807 time series. To enable robust pretraining, we construct the first observability-specific pretraining corpus derived entirely from production environments and introduce a hybrid pretraining strategy combining real, open-source, and synthetic time-series data. Evaluated on BOOM and multiple general-purpose time-series benchmarks, Toto achieves state-of-the-art performance. All model weights, inference code, and evaluation scripts are publicly released under the Apache 2.0 license, facilitating standardized, reproducible research in observability-driven time-series modeling.

Technology Category

• Machine Learning: Time-Series/Data Streams
• Computer Vision: Large Vision Models
• Natural Language Processing: (Large) Language Models

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

We introduce Toto, a time series forecasting foundation model with 151 million parameters. Toto uses a modern decoder-only architecture coupled with architectural innovations designed to account for specific challenges found in multivariate observability time series data. Toto's pre-training corpus is a mixture of observability data, open datasets, and synthetic data, and is 4-10$ imes$ larger than those of leading time series foundation models. Additionally, we introduce BOOM, a large-scale benchmark consisting of 350 million observations across 2,807 real-world time series. For both Toto and BOOM, we source observability data exclusively from Datadog's own telemetry and internal observability metrics. Extensive evaluations demonstrate that Toto achieves state-of-the-art performance on both BOOM and on established general purpose time series forecasting benchmarks. Toto's model weights, inference code, and evaluation scripts, as well as BOOM's data and evaluation code, are all available as open source under the Apache 2.0 License available at https://huggingface.co/Datadog/Toto-Open-Base-1.0 and https://github.com/DataDog/toto.

Problem

• Develops Toto, a large-scale time series forecasting foundation model
• Addresses challenges in multivariate observability time series data
• Introduces BOOM benchmark for evaluating time series models

Innovation

• Decoder-only architecture for time series
• Largest pre-training corpus for observability
• Open-source model weights and benchmark