Speaking Numbers to LLMs: Multi-Wavelet Number Embeddings for Time Series Forecasting

2026-06-24

AI Summary

Large language models (LLMs) struggle to effectively model continuous numerical time series due to their discrete tokenization mechanisms, which incur losses in both precision and sequential fidelity. To address this limitation, this work proposes TempoWave—a plug-and-play multi-wavelet numerical embedding interface that introduces multi-resolution wavelet analysis into LLM-based numerical representation for the first time. By generating bit-level embeddings from multi-scale wavelet coefficients, TempoWave unifies the modeling of numeric format, bit identity, and normalization robustness. This approach directly replaces standard tokens within the Transformer architecture, enabling simultaneous capture of local fluctuations and global temporal structure. Evaluated across five context-rich time series forecasting benchmarks, TempoWave significantly outperforms existing numerical tokenization methods, establishing a new state-of-the-art.

Abstract

Large language models (LLMs) are attractive for context-aware time series forecasting because they can integrate heterogeneous textual signals, yet their discrete, language-oriented tokenization and embedding interfaces are misaligned with continuous numerical values, often harming numerical ordering and forecasting reliability. We propose TempoWave, a plug-and-play temporal wavelet digit interface that maps each scalar observation into digit-wise embeddings constructed from multi-wavelet, multi-scale coefficients. By directly overriding standard token representations, TempoWave seamlessly exposes both fine-grained local fluctuations and macro global structures in a transformer-compatible form, ensuring that precise numerical formatting, distinct digit identity, and robustness to common normalization operations are maintained throughout the LLM pipeline. Experiments across five context-enriched forecasting benchmarks demonstrate that TempoWave consistently improves LLM-based forecasters over standard numeric tokenization and alternative embedding interfaces, achieving a new state-of-the-art. These results highlight the numeric interface as a key bottleneck and suggest that principled multi-resolution embeddings can better couple LLMs' contextual reasoning with precise forecasting. Our code is available at https://github.com/DC-research/TempoWAVE and our model can be accessed at https://huggingface.co/Melady/TempoWAVE.

Problem

• large language models
• time series forecasting
• numerical embedding
• tokenization
• numeric interface

Innovation

• multi-wavelet embeddings
• time series forecasting
• large language models
• numeric tokenization
• plug-and-play interface