Deep TPC: Temporal-Prior Conditioning for Time Series Forecasting

2026-02-18

AI Summary

This work addresses the limitation of existing large language model (LLM)-based time series methods, which typically employ shallow modeling of temporal information and thus struggle with deep temporal reasoning. To overcome this, the authors propose Temporal-Prior Conditioning (TPC), a novel approach that treats time as a first-class modality by dynamically injecting learnable temporal tokens at multiple network depths. These tokens explicitly disentangle temporal signals from contextual content through cross-attention interactions with compact time descriptions encoded by a frozen LLM. By training only a lightweight cross-attention module, TPC achieves state-of-the-art long-term forecasting performance across diverse datasets, significantly outperforming both full fine-tuning and shallow conditioning strategies while maintaining low parameter overhead.

Technology Category

• Machine Learning: Time-Series/Data Streams
• Natural Language Processing: (Large) Language Models
• Planning, Routing, and Scheduling: Temporal Planning

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

LLM-for-time series (TS) methods typically treat time shallowly, injecting positional or prompt-based cues once at the input of a largely frozen decoder, which limits temporal reasoning as this information degrades through the layers. We introduce Temporal-Prior Conditioning (TPC), which elevates time to a first-class modality that conditions the model at multiple depths. TPC attaches a small set of learnable time series tokens to the patch stream; at selected layers these tokens cross-attend to temporal embeddings derived from compact, human-readable temporal descriptors encoded by the same frozen LLM, then feed temporal context back via self-attention. This disentangles time series signal and temporal information while maintaining a low parameter budget. We show that by training only the cross-attention modules and explicitly disentangling time series signal and temporal information, TPC consistently outperforms both full fine-tuning and shallow conditioning strategies, achieving state-of-the-art performance in long-term forecasting across diverse datasets. Code available at: https://github.com/fil-mp/Deep_tpc

Problem

• time series forecasting
• temporal reasoning
• large language models
• temporal information degradation

Innovation

• Temporal-Prior Conditioning
• time series forecasting
• large language models
• cross-attention
• temporal disentanglement