Large Language Models are Few-shot Multivariate Time Series Classifiers

2025-01-30

AI Summary

To address the challenge of few-shot multivariate time-series classification in industrial settings, this paper proposes LLMFew—a novel framework that pioneers the adaptation of large language models (LLMs) to time-series classification tasks. It introduces a Patch-based Temporal Convolutional Encoder (PTCEnc) to achieve structured alignment from multivariate time-series data into the LLM’s text embedding space, and incorporates a LoRA-enhanced lightweight decoder to strengthen temporal representation learning. Evaluated on Handwriting and EthanolConcentration datasets, LLMFew achieves accuracy improvements of 125.2% and 50.2%, respectively, significantly outperforming existing state-of-the-art methods. Extensive experiments demonstrate the robustness and generalization capability of LLMs under data-scarce industrial conditions. This work establishes a new paradigm for time-series understanding and few-shot learning by bridging foundation models with temporal data modeling.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Computer Vision: Large Vision Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Large Language Models (LLMs) have been extensively applied in time series analysis. Yet, their utility in the few-shot classification (i.e., a crucial training scenario due to the limited training data available in industrial applications) concerning multivariate time series data remains underexplored. We aim to leverage the extensive pre-trained knowledge in LLMs to overcome the data scarcity problem within multivariate time series. Specifically, we propose LLMFew, an LLM-enhanced framework to investigate the feasibility and capacity of LLMs for few-shot multivariate time series classification. This model introduces a Patch-wise Temporal Convolution Encoder (PTCEnc) to align time series data with the textual embedding input of LLMs. We further fine-tune the pre-trained LLM decoder with Low-rank Adaptations (LoRA) to enhance its feature representation learning ability in time series data. Experimental results show that our model outperformed state-of-the-art baselines by a large margin, achieving 125.2% and 50.2% improvement in classification accuracy on Handwriting and EthanolConcentration datasets, respectively. Moreover, our experimental results demonstrate that LLM-based methods perform well across a variety of datasets in few-shot MTSC, delivering reliable results compared to traditional models. This success paves the way for their deployment in industrial environments where data are limited.

Problem

• Limited Data
• Multivariate Time Series
• Large Language Model Performance

Innovation

• LLMFew Model
• Temporal Data Classification
• Few-shot Learning