Beyond Tokenization: Direct Timestep Embedding and Contrastive Alignment for Time-Series Question Answering

2026-06-17

AI Summary

This work addresses the limitations of conventional approaches that introduce numerical distortion, loss of temporal precision, and disruption of trend structures when encoding time series into large language models through tokenization and fixed-window strategies. To overcome these issues, the authors propose the CADE framework, which eschews tokenization and chunking entirely. Instead, it employs a point-wise linear encoder coupled with an MLP projector to generate temporally precise, step-level embeddings. Furthermore, a unidirectional supervised contrastive loss anchored on class-name prompts is introduced to align time series representations with textual semantics. The proposed method demonstrates substantially improved cross-dataset generalization, consistently outperforming both open-source and closed-source large language model baselines across all six time series question-answering tasks in the Time-MQA benchmark.

Abstract

Recent advances in large language models (LLMs) have given rise to time-series question answering (TSQA), which formulates time-series analysis as natural-language question answering. However, directly feeding raw numerical series into LLMs suffers from a tokenization bottleneck: Byte Pair Encoding fragments continuous values into unstable tokens whose embeddings lack meaningful metric structure, resulting in the loss of magnitude, scale, and trend information. Prior methods use patch-based encoders that split the series into fixed windows, locking in one granularity that breaks patterns and hides exact timesteps, through a separate module that rarely transfers across datasets with different lengths or sampling rates. To address this challenge, we propose CADE (Contrastive Alignment with Direct Embedding), a novel framework for TSQA built upon two key components: direct timestep embedding and semantic alignment. The proposed framework maps each timestep directly into the LLM embedding space through a point-wise linear encoder and MLP projector, preserving exact index-level access while eliminating the need for patching and padding. To further bridge the semantic gap between time-series and language representations, we introduce a novel one-directional supervised contrastive loss that aligns time-series embeddings with frozen class-name text anchors. Experimental results on the public Time-MQA benchmark demonstrate that our framework consistently improves performance across six TSQA tasks, outperforming both open-source and proprietary LLM baselines.

Problem

• time-series question answering
• tokenization bottleneck
• timestep embedding
• semantic alignment
• large language models

Innovation

• Direct Timestep Embedding
• Contrastive Alignment
• Time-Series Question Answering
• Tokenization-Free Representation
• Supervised Contrastive Learning