TS-Skill: A Benchmark for Evaluating Analytical Skills in Time-Series Question Answering

2026-05-23

AI Summary

Existing time series question answering (TSQA) benchmarks struggle to evaluate models’ fine-grained temporal reasoning capabilities at the signal level. To address this limitation, this work proposes TS-Skill, the first skill-decomposed evaluation framework for TSQA, focusing on three composable core skills: temporal scale selection, temporal localization, and cross-interval integration. Leveraging the SKEvol agent framework, the benchmark is constructed through domain-aware seed generation, skill-controlled question synthesis, code-assisted answer generation, and multi-stage signal alignment verification, enabling large-scale, high-quality data creation. Experiments across ten prominent large language models and time-series language models reveal significant and uneven performance gaps in these three skills, with cross-interval integration emerging as the most challenging capability.

Abstract

Large language models (LLMs) and time-series language models (TSLMs) are increasingly applied to time-series question answering (TSQA). Unlike text-only QA, TSQA requires models to ground answers in temporal signals whose patterns may occur at different scales, specific time locations, or across separated intervals. However, existing benchmarks are typically organized by task types or high-level reasoning categories, making it difficult to diagnose the underlying signal-level capabilities driving model performance. We introduce TS-Skill, a controlled benchmark for evaluating three composable analytical skills in TSQA: temporal scale selection (SK1), temporal localization (SK2), and cross-interval integration (SK3). TS-Skill provides timestamp-aware questions, broad domain coverage, and human-validated QA quality. To construct the benchmark at scale, we develop SKEvol, a skill-guided agentic framework that combines domain-aware time-series seed generation, skill-controlled question generation, metadata- and code-assisted answer construction, multi-phase signal-grounded verification, and human-in-the-loop curation. Experiments on ten state-of-the-art LLMs and TSLMs reveal substantial and uneven capability gaps across SK1-SK3. In particular, SK3 remains consistently challenging for non-agent models, whereas tool-augmented agents show a selective advantage on standalone SK3. These findings demonstrate that skill-level evaluation can uncover temporal reasoning failures that are obscured by aggregate TSQA scores.

Problem

• time-series question answering
• analytical skills
• temporal reasoning
• benchmark evaluation
• signal-level capabilities

Innovation

• time-series question answering
• analytical skill benchmark
• skill-guided agent framework
• temporal reasoning evaluation
• cross-interval integration