TimeCAP: Learning to Contextualize, Augment, and Predict Time Series Events with Large Language Model Agents

2025-02-17

AI Summary

To address insufficient contextual understanding in time-series event prediction, this paper proposes a dual-LLM agent collaboration framework: the first agent encodes raw time-series data into a semantic summary, while the second agent performs event prediction based on this summary. We innovatively introduce a multimodal encoder that enables mutual enhancement between raw inputs and contextual examples, supporting dynamic in-context example injection. This approach breaks the prevailing paradigm of using LLMs solely as end-to-end predictors, instead decoupling their role into two distinct functions—context modeling and decision-based prediction. Evaluated on multiple real-world datasets, our method achieves an average 28.75% improvement in F1 score over state-of-the-art baselines, including both directly fine-tuned and prompt-engineered LLM approaches.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Time series data is essential in various applications, including climate modeling, healthcare monitoring, and financial analytics. Understanding the contextual information associated with real-world time series data is often essential for accurate and reliable event predictions. In this paper, we introduce TimeCAP, a time-series processing framework that creatively employs Large Language Models (LLMs) as contextualizers of time series data, extending their typical usage as predictors. TimeCAP incorporates two independent LLM agents: one generates a textual summary capturing the context of the time series, while the other uses this enriched summary to make more informed predictions. In addition, TimeCAP employs a multi-modal encoder that synergizes with the LLM agents, enhancing predictive performance through mutual augmentation of inputs with in-context examples. Experimental results on real-world datasets demonstrate that TimeCAP outperforms state-of-the-art methods for time series event prediction, including those utilizing LLMs as predictors, achieving an average improvement of 28.75% in F1 score.

Problem

• Enhance time series event prediction accuracy
• Contextualize time series using LLMs
• Integrate multi-modal data for improved predictions

Innovation

• LLM agents for context
• Multi-modal encoder integration
• Enhanced time series prediction