BSAT: B-Spline Adaptive Tokenizer for Long-Term Time Series Forecasting

2026-01-02

arXiv.org

AI Summary

This work addresses the high computational complexity of self-attention in Transformers for long-term time series forecasting and the mismatch between fixed-length tokenization and the intrinsic semantic structure of time series data. To this end, the authors propose a parameter-free, B-spline-based adaptive tokenization method that dynamically generates non-uniform tokens in high-curvature regions by fitting B-spline curves, thereby yielding a semantics-aware compressed representation. Furthermore, they introduce a hybrid positional encoding scheme combining learnable embeddings with a layer-wise learnable basis for L-RoPE, enhancing the model’s capacity to capture multi-scale temporal dependencies. Experimental results demonstrate that the proposed approach achieves competitive performance across multiple public benchmarks, with particularly strong results under high compression ratios and memory-constrained settings.

Technology Category

• Machine Learning: Time-Series/Data Streams
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs

Abstract

Long-term time series forecasting using transformers is hampered by the quadratic complexity of self-attention and the rigidity of uniform patching, which may be misaligned with the data's semantic structure. In this paper, we introduce the \textit{B-Spline Adaptive Tokenizer (BSAT)}, a novel, parameter-free method that adaptively segments a time series by fitting it with B-splines. BSAT algorithmically places tokens in high-curvature regions and represents each variable-length basis function as a fixed-size token, composed of its coefficient and position. Further, we propose a hybrid positional encoding that combines a additive learnable positional encoding with Rotary Positional Embedding featuring a layer-wise learnable base: L-RoPE. This allows each layer to attend to different temporal dependencies. Our experiments on several public benchmarks show that our model is competitive with strong performance at high compression rates. This makes it particularly well-suited for use cases with strong memory constraints.

Problem

• long-term time series forecasting
• self-attention complexity
• uniform patching
• semantic structure alignment
• time series tokenization

Innovation

• B-Spline
• Adaptive Tokenization
• Long-term Time Series Forecasting
• Rotary Positional Embedding
• Parameter-free Segmentation