From Recognition to Understanding: Unlocking Cognitive Time Series Reasoning with LLMs

2026-06-20

AI Summary

This work addresses the limited support for high-level semantic understanding and cognitive reasoning in existing approaches that integrate time series with large language models (LLMs), which predominantly focus on low-level forecasting. To bridge this gap, we introduce TSCognition, the first multimodal benchmark tailored for temporal cognitive reasoning, comprising 41K question-answer pairs from 15 diverse sources and spanning five task types: decoding, anchoring, reasoning, extrapolation, and action. We further propose TSAlign, a lightweight unified framework that efficiently aligns time series with LLM semantic spaces through compact segment representations, gated residual injection, and multivariate fusion. Experiments demonstrate that TSAlign substantially outperforms current LLMs, vision-language models, and time-series question-answering methods on both TSCognition and TimerBed, while significantly reducing computational overhead.

Abstract

Time series analysis has recently been coupled with Large Language Models (LLMs) to leverage their reasoning and world knowledge capabilities, yet gains remain limited. We attribute this to a fundamental mismatch between existing task formulations and LLM strengths: most settings reduce time series understanding to curve-fitting systems, focusing on low-level prediction while ignoring the semantic, contextual, and reasoning-intensive nature of real-world temporal decision-making.To address these limitations, we introduce TSCognition, a multimodal benchmark for multi-dimensional time series reasoning. It collects real-world time series and textual information from 15 public sources and constructs approximately 41K QA samples around five cognitive reasoning tasks: Decoding, Grounding, Inferring, Extrapolating, and Acting. Building on this, we further propose TSAlign, a unified framework that encodes time series into compact patch-level representations and aligns them with semantic directions in the LLM embedding space via gated residual injection and multivariate fusion.Experiments show that TSAlign outperforms existing LLM, VLM, and time series QA baselines on TSCognition and the publicly available TimerBed benchmark while substantially reducing computational cost.Code is available at: [https://github.com/EIT-NLP/CognitiveTSR](https://github.com/EIT-NLP/CognitiveTSR)

Problem

• time series reasoning
• Large Language Models
• cognitive understanding
• semantic context
• temporal decision-making

Innovation

• time series reasoning
• large language models
• multimodal alignment
• cognitive benchmark
• patch-level representation