Context-Alignment: Activating and Enhancing LLM Capabilities in Time Series

2025-01-07

AI Summary

Existing methods feed time series into large language models (LLMs) solely via token-level matching, neglecting LLMs’ inherent capacity for linguistic logic and structural reasoning—thereby limiting temporal understanding. This paper introduces a novel context-alignment paradigm that maps time-series data into linguistically coherent, LLM-native contextual representations through dual alignment along structural and logical dimensions. Key contributions include: (1) the first context-level (rather than token-level) alignment mechanism; (2) a Structure–Logic Dual Alignment framework with the DECA prompting strategy; and (3) a hierarchical, plug-and-play LLM-enhancement architecture integrating DSCA-GNNs, time-series–language multimodal modeling, and modular LLM adaptation. Experiments demonstrate significant improvements over state-of-the-art methods on few-shot and zero-shot time-series forecasting tasks. Moreover, context alignment serves as a strong inductive bias, enhancing LLMs’ holistic long-horizon temporal modeling and generalization capability.

Technology Category

• Machine Learning: Time-Series/Data Streams
• Natural Language Processing: (Large) Language Models
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Recently, leveraging pre-trained Large Language Models (LLMs) for time series (TS) tasks has gained increasing attention, which involves activating and enhancing LLMs' capabilities. Many methods aim to activate LLMs' capabilities based on token-level alignment but overlook LLMs' inherent strength on natural language processing -- their deep understanding of linguistic logic and structure rather than superficial embedding processing. We propose Context-Alignment, a new paradigm that aligns TS with a linguistic component in the language environments familiar to LLMs to enable LLMs to contextualize and comprehend TS data, thereby activating their capabilities. Specifically, such context-level alignment comprises structural alignment and logical alignment, which is achieved by a Dual-Scale Context-Alignment GNNs (DSCA-GNNs) applied to TS-language multimodal inputs. Structural alignment utilizes dual-scale nodes to describe hierarchical structure in TS-language, enabling LLMs treat long TS data as a whole linguistic component while preserving intrinsic token features. Logical alignment uses directed edges to guide logical relationships, ensuring coherence in the contextual semantics. Demonstration examples prompt are employed to construct Demonstration Examples based Context-Alignment (DECA) following DSCA-GNNs framework. DECA can be flexibly and repeatedly integrated into various layers of pre-trained LLMs to improve awareness of logic and structure, thereby enhancing performance. Extensive experiments show the effectiveness of DECA and the importance of Context-Alignment across tasks, particularly in few-shot and zero-shot forecasting, confirming that Context-Alignment provide powerful prior knowledge on context.

Problem

• Time Series Data
• Language Models
• Semantic Understanding

Innovation

• Context Alignment
• Neural Network
• Time Series Data