TiRex: Zero-Shot Forecasting Across Long and Short Horizons with Enhanced In-Context Learning

2025-05-29

AI Summary

Current large language models exhibit limited performance in zero-shot time series forecasting: Transformers suffer from weak long-range dependency modeling, while LSTMs lack in-context learning capability. To address this, TiRex introduces the first integrated architecture combining extended LSTM (xLSTM) with context-aware prompting masking (CPM), jointly enabling robust state tracking and in-context learning. It further proposes a sequence-to-sequence encoding scheme and a zero-shot prompting inference mechanism, supporting joint modeling of short- and long-horizon forecasts without fine-tuning and enabling generalization to unseen sequences. Evaluated on the GiftEval and Chronos-ZS benchmarks, TiRex consistently outperforms state-of-the-art models—including TabPFN-TS, Chronos Bolt, TimesFM, and Moirai—despite using significantly fewer parameters, thereby establishing new performance records for zero-shot time series forecasting.

Technology Category

• Machine Learning: Time-Series/Data Streams
• Natural Language Processing: (Large) Language Models
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

In-context learning, the ability of large language models to perform tasks using only examples provided in the prompt, has recently been adapted for time series forecasting. This paradigm enables zero-shot prediction, where past values serve as context for forecasting future values, making powerful forecasting tools accessible to non-experts and increasing the performance when training data are scarce. Most existing zero-shot forecasting approaches rely on transformer architectures, which, despite their success in language, often fall short of expectations in time series forecasting, where recurrent models like LSTMs frequently have the edge. Conversely, while LSTMs are well-suited for time series modeling due to their state-tracking capabilities, they lack strong in-context learning abilities. We introduce TiRex that closes this gap by leveraging xLSTM, an enhanced LSTM with competitive in-context learning skills. Unlike transformers, state-space models, or parallelizable RNNs such as RWKV, TiRex retains state-tracking, a critical property for long-horizon forecasting. To further facilitate its state-tracking ability, we propose a training-time masking strategy called CPM. TiRex sets a new state of the art in zero-shot time series forecasting on the HuggingFace benchmarks GiftEval and Chronos-ZS, outperforming significantly larger models including TabPFN-TS (Prior Labs), Chronos Bolt (Amazon), TimesFM (Google), and Moirai (Salesforce) across both short- and long-term forecasts.

Problem

• Enhancing zero-shot time series forecasting with in-context learning
• Bridging the gap between LSTM state-tracking and in-context learning
• Outperforming existing models in short- and long-term forecasting benchmarks

Innovation

• Uses xLSTM for enhanced in-context learning
• Incorporates CPM masking for state-tracking
• Achieves SOTA in zero-shot time series forecasting