Time Series Language Model for Descriptive Caption Generation

2025-01-03

AI Summary

Time-series data often lack high-quality natural language descriptions, and existing pretrained models suffer from limited interpretability due to the scarcity of fine-grained temporal annotations. Method: This paper proposes the Temporal Semantic Language Model (TSLM), the first encoder-decoder architecture jointly modeling time-series embeddings and textual representations. It introduces a context-aware prompt-driven synthetic data generation mechanism to alleviate annotation bottlenecks, and incorporates cross-modal dense retrieval scoring with denoising strategies to enhance description accuracy and temporal sensitivity. Contribution/Results: Evaluated on a multi-source time-series captioning benchmark, TSLM significantly outperforms state-of-the-art methods. It demonstrates strong cross-domain generalization and high fidelity in natural language generation, enabling precise, semantically rich, and temporally grounded descriptions of time-series patterns.

Technology Category

• Natural Language Processing: Generation
• Machine Learning: Time-Series/Data Streams
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

The automatic generation of representative natural language descriptions for observable patterns in time series data enhances interpretability, simplifies analysis and increases cross-domain utility of temporal data. While pre-trained foundation models have made considerable progress in natural language processing (NLP) and computer vision (CV), their application to time series analysis has been hindered by data scarcity. Although several large language model (LLM)-based methods have been proposed for time series forecasting, time series captioning is under-explored in the context of LLMs. In this paper, we introduce TSLM, a novel time series language model designed specifically for time series captioning. TSLM operates as an encoder-decoder model, leveraging both text prompts and time series data representations to capture subtle temporal patterns across multiple phases and generate precise textual descriptions of time series inputs. TSLM addresses the data scarcity problem in time series captioning by first leveraging an in-context prompting synthetic data generation, and second denoising the generated data via a novel cross-modal dense retrieval scoring applied to time series-caption pairs. Experimental findings on various time series captioning datasets demonstrate that TSLM outperforms existing state-of-the-art approaches from multiple data modalities by a significant margin.

Problem

• Time Series Analysis
• Pre-trained Models
• Data Scarcity

Innovation

• TSLM
• Data Augmentation
• High-quality Sample Selection