Can LLM Coding Agents Reason About Time Series?

2026-06-15

AI Summary

This study systematically evaluates the temporal reasoning capabilities of large language models (LLMs) in domains such as finance, healthcare, and environmental monitoring. It presents the first comparative analysis between two paradigms: direct processing of raw numerical time series and using LLMs as coding agents that generate executable Python code. A strong LLM-based adjudicator is introduced to conduct in-depth analysis of reasoning strategies and failure modes. Leveraging two established benchmarks and combining statistical testing with qualitative assessment, the experiments reveal that the coding-agent approach improves accuracy by up to 10% over pure textual input for comprehension tasks, yet overall error rates remain substantial at 22–34%. While LLMs often select appropriate statistical methods, they frequently overlook critical details. This work establishes the first systematic empirical framework and provides key insights into deploying LLMs for time series analysis.

Abstract

Large language models (LLMs) are increasingly being used for automated decision-making systems in finance, healthcare, or environmental monitoring. Time series data are ubiquitous in these fields, yet hard to process automatically. Can time series be analyzed by LLM agents? We examine three approaches: providing the agent with raw numerical data, using the LLM as a coding agent, or a combination of both. In the coding agent setup, the model iteratively queries the data using Python code. Using two time series understanding benchmarks, we show that agents with code access can outperform models processing raw data by up to 10%. However, even the best performing agent still answers about 22-34% of the questions incorrectly. To get insights into models' strategies and reasoning gaps, we analyze the model outputs with a strong LLM judge. Our analysis reveals that coding agents can select appropriate statistical tests, but often miss important nuances. Meanwhile, models with access to raw data can reach the right conclusions using back-of-the-envelope calculations.

Problem

• time series
• large language models
• reasoning
• coding agents
• automated decision-making

Innovation

• LLM coding agents
• time series reasoning
• code-based querying
• automated data analysis
• LLM evaluation