TS-MLLM: A Multi-Modal Large Language Model-based Framework for Industrial Time-Series Big Data Analysis

2026-03-08

AI Summary

This work addresses the limitations of existing industrial time-series analysis methods, which are predominantly confined to unimodal modeling and struggle to effectively integrate time-domain signals, frequency-domain images, and textual knowledge—thereby constraining predictive performance in equipment health monitoring. To overcome this, we propose the first tri-modal large language model framework tailored for industrial time-series data. Our approach leverages temporal chunking, a spectrum-aware vision-language alignment mechanism (SVLMA), and a temporal-centric multimodal attention fusion (TMAF) module to achieve deep alignment and collaborative modeling across time-series, spectral images, and text. Evaluated on multiple industrial benchmarks, the proposed method significantly outperforms state-of-the-art techniques, demonstrating superior robustness, generalization, and computational efficiency—particularly under few-shot and complex operational scenarios.

Technology Category

• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Accurate analysis of industrial time-series big data is critical for the Prognostics and Health Management (PHM) of industrial equipment. While recent advancements in Large Language Models (LLMs) have shown promise in time-series analysis, existing methods typically focus on single-modality adaptations, failing to exploit the complementary nature of temporal signals, frequency-domain visual representations, and textual knowledge information. In this paper, we propose TS-MLLM, a unified multi-modal large language model framework designed to jointly model temporal signals, frequency-domain images, and textual domain knowledge. Specifically, we first develop an Industrial time-series Patch Modeling branch to capture long-range temporal dynamics. To integrate cross-modal priors, we introduce a Spectrum-aware Vision-Language Model Adaptation (SVLMA) mechanism that enables the model to internalize frequency-domain patterns and semantic context. Furthermore, a Temporal-centric Multi-modal Attention Fusion (TMAF) mechanism is designed to actively retrieve relevant visual and textual cues using temporal features as queries, ensuring deep cross-modal alignment. Extensive experiments on multiple industrial benchmarks demonstrate that TS-MLLM significantly outperforms state-of-the-art methods, particularly in few-shot and complex scenarios. The results validate our framework's superior robustness, efficiency, and generalization capabilities for industrial time-series prediction.

Problem

• Industrial Time-Series
• Multi-Modal Learning
• Large Language Models
• Prognostics and Health Management
• Cross-Modal Fusion

Innovation

• Multi-modal Large Language Model
• Time-Series Analysis
• Spectrum-aware Vision-Language Adaptation
• Temporal-centric Attention Fusion
• Industrial Prognostics and Health Management