Unveiling the Potential of Text in High-Dimensional Time Series Forecasting

2025-01-13

AI Summary

This paper addresses the curse of dimensionality and data sparsity in high-dimensional time series forecasting by systematically integrating large language models (LLMs) to encode textual prior knowledge—introducing a novel text–time-series bimodal collaborative forecasting framework. Methodologically, it adopts a dual-tower architecture: one tower employs an LLM encoder to process descriptive textual inputs (e.g., metric definitions, event annotations), while the other applies a time-series feature extractor to model numerical sequences; cross-modal representations are fused via linear projection for joint representation learning. Unlike conventional purely numerical modeling paradigms, this framework explicitly leverages semantic textual cues to augment temporal modeling. Empirical evaluation on multiple high-dimensional benchmark datasets demonstrates substantial improvements in forecasting accuracy, validating the critical role of textual semantics in mitigating data sparsity and enhancing model robustness. The work establishes a new paradigm for multimodal time series forecasting.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Data Mining & Knowledge Management: Mining of Visual, Multimedia & Multimodal Data

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Time series forecasting has traditionally focused on univariate and multivariate numerical data, often overlooking the benefits of incorporating multimodal information, particularly textual data. In this paper, we propose a novel framework that integrates time series models with Large Language Models to improve high-dimensional time series forecasting. Inspired by multimodal models, our method combines time series and textual data in the dual-tower structure. This fusion of information creates a comprehensive representation, which is then processed through a linear layer to generate the final forecast. Extensive experiments demonstrate that incorporating text enhances high-dimensional time series forecasting performance. This work paves the way for further research in multimodal time series forecasting.

Problem

• Complex Time Series Prediction
• Text Information Integration
• Accuracy Improvement

Innovation

• Time Series Analysis
• Multi-modal Fusion
• Language Model Integration