Baguan-TS: A Sequence-Native In-Context Learning Model for Time Series Forecasting with Covariates

2026-03-18

AI Summary

This work proposes Baguan-TS, the first native sequence-based in-context learning framework for covariate time series forecasting. Existing approaches either rely on handcrafted features or lack gradient-free rapid adaptation at inference time, struggling to balance end-to-end modeling with efficient generalization. Baguan-TS addresses these limitations through a unified three-dimensional Transformer architecture that jointly models temporal, variate, and contextual dimensions. It further incorporates target-space retrieval calibration and a context overfitting mechanism to mitigate training instability and excessive output smoothing. Evaluated across multiple public benchmarks and real-world energy datasets, Baguan-TS consistently achieves state-of-the-art performance and robustness in both point and probabilistic forecasting tasks.

Technology Category

• Machine Learning: Time-Series/Data Streams
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Transformers enable in-context learning (ICL) for rapid, gradient-free adaptation in time series forecasting, yet most ICL-style approaches rely on tabularized, hand-crafted features, while end-to-end sequence models lack inference-time adaptation. We bridge this gap with a unified framework, Baguan-TS, which integrates the raw-sequence representation learning with ICL, instantiated by a 3D Transformer that attends jointly over temporal, variable, and context axes. To make this high-capacity model practical, we tackle two key hurdles: (i) calibration and training stability, improved with a feature-agnostic, target-space retrieval-based local calibration; and (ii) output oversmoothing, mitigated via context-overfitting strategy. On public benchmark with covariates, Baguan-TS consistently outperforms established baselines, achieving the highest win rate and significant reductions in both point and probabilistic forecasting metrics. Further evaluations across diverse real-world energy datasets demonstrate its robustness, yielding substantial improvements.

Problem

• in-context learning
• time series forecasting
• sequence modeling
• covariates
• adaptation gap

Innovation

• In-Context Learning
• Time Series Forecasting
• 3D Transformer
• Sequence-Native Representation
• Context-Overfitting