Continuity and Ordinality Matter: Constraining Time Series Tokens for Effective Time Series Analysis with Large Language Models

2026-05-22

AI Summary

This work addresses a critical limitation in existing token-based large language models for time series, which neglect the inherent continuity and ordinality of time series tokens, thereby constraining modeling performance. The study proposes the first systematic approach to explicitly encode these structural priors as geometric constraints in the embedding space, preserving continuity and ordinal relationships both during initialization and end-to-end training. By incorporating such inductive biases, the method achieves substantial performance gains across multiple time series forecasting and analysis benchmarks, demonstrating superior generalization capabilities and competitive results compared to state-of-the-art approaches.

Abstract

Token-based time series large language models (TS-LLMs) have emerged as a promising direction for time series analysis and reasoning. However, prior studies largely overlook the inherent continuity and ordinality of time series tokens, which substantially limits model performance. In this paper, we argue that preserving these properties in time series token embeddings is crucial for the effectiveness of token-based TS-LLMs. To this end, we propose COM (Continuity and Ordinality Matter), a continuity- and ordinality-aware strategy that integrates geometric constraints into both the initialization and training stages. Empirical results on multiple time series analysis benchmarks demonstrate that COM consistently improves the performance of token-based TS-LLMs, achieving competitive results and strong generalizability. Code is available at https://anonymous.4open.science/r/COM .

Problem

• time series
• large language models
• continuity
• ordinality
• token-based

Innovation

• continuity
• ordinality
• time series tokens
• geometric constraints
• large language models