Towards Verifiable Agentic Data Science: Solving Irregular TSQA Via Tool-Grounded Reasoning

2026-06-13

AI Summary

This work addresses the limitation of existing time series question answering (TSQA) benchmarks, which predominantly rely on regularly sampled data and thus fail to evaluate large language models (LLMs) in realistic irregular temporal settings. To bridge this gap, the authors present IRTS-ToolBench, the first open benchmark specifically designed for irregular TSQA. It encompasses 1,700 questions across 13 domains and 10 task types, explicitly incorporating real-world challenges such as asynchronous observations, informative missing values, and variable-frequency sampling. The benchmark supports tool-augmented reasoning by providing a standardized input format, modular tool integration interfaces, and a reproducible evaluation protocol, thereby establishing a unified testbed for assessing the reliability of LLMs in practical time series applications.

Abstract

Time series data in real-world deployments is overwhelmingly irregular. Observations are asynchronous, missing values are informative rather than random, and sampling frequencies vary across sensors and operational windows. However, existing Time Series Question Answering (TSQA) benchmarks mostly assume regularly sampled inputs, leaving a fundamental gap in understanding how large language models (LLMs) and AI agents perform under irregular conditions. To bridge this gap, we introduce IRTS-ToolBench, a benchmark of 1,700 questions spanning 10 task types across 13 domains. IRTS-ToolBench is designed to be used independently by any researcher working on LLM-based irregular time series analysis, providing standardized inputs and a reproducible evaluation protocol. Code can be found in https://github.com/SanhornC/IRTS-ToolBench.

Problem

• irregular time series
• Time Series Question Answering
• LLM evaluation
• asynchronous observations
• missing data

Innovation

• Irregular Time Series
• Time Series Question Answering
• Tool-Grounded Reasoning
• LLM-based Agents
• Benchmark