Chain-of-thought Reviewing and Correction for Time Series Question Answering

2025-12-27

AI Summary

To address the vulnerability of large language models (LLMs) to numerical reasoning errors in time-series question answering (TSQA), this paper proposes T3LLM—a tri-role collaborative framework comprising Worker, Reviewer, and Student agents. The framework enables chain-of-thought (CoT) generation, consistency verification against raw time-series data, and error-driven capability internalization. We introduce a novel, verifiable CoT review mechanism specifically designed for time-series data, wherein review feedback is explicitly transformed into optimization signals for the Student model. Furthermore, error-correcting supervised fine-tuning enhances reasoning robustness. Evaluated on multiple real-world TSQA benchmarks, T3LLM achieves state-of-the-art performance: up to 18.7% improvement in reasoning accuracy and 92.4% success rate in identifying and correcting erroneous reasoning steps.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

With the advancement of large language models (LLMs), diverse time series analysis tasks are reformulated as time series question answering (TSQA) through a unified natural language interface. However, existing LLM-based approaches largely adopt general natural language processing techniques and are prone to reasoning errors when handling complex numerical sequences. Different from purely textual tasks, time series data are inherently verifiable, enabling consistency checking between reasoning steps and the original input. Motivated by this property, we propose T3LLM, which performs multi-step reasoning with an explicit correction mechanism for time series question answering. The T3LLM framework consists of three LLMs, namely, a worker, a reviewer, and a student, that are responsible for generation, review, and reasoning learning, respectively. Within this framework, the worker generates step-wise chains of thought (CoT) under structured prompts, while the reviewer inspects the reasoning, identifies erroneous steps, and provides corrective comments. The collaboratively generated corrected CoT are used to fine-tune the student model, internalizing multi-step reasoning and self-correction into its parameters. Experiments on multiple real-world TSQA benchmarks demonstrate that T3LLM achieves state-of-the-art performance over strong LLM-based baselines.

Problem

• Addresses reasoning errors in time series question answering
• Introduces a correction mechanism for multi-step reasoning
• Enhances LLM performance on complex numerical sequence tasks

Innovation

• Multi-agent LLM framework with worker, reviewer, student
• Structured prompts generate step-wise chains of thought
• Reviewer identifies errors and provides corrective comments