VLT: A Vision-Language-Time Series Multimodal Foundation Model for Industrial Intelligence

2026-07-15

AI Summary

This work addresses the challenge of effectively fusing time-series signals with textual semantics in industrial AI and the limited generalization of unimodal models by proposing VLT, a novel multimodal foundation model. VLT innovatively leverages spectral representations as a visual bridge connecting continuous time-series and discrete text, and introduces a Time-MoE module to capture heterogeneous temporal dynamics. Cross-modal synergy is achieved through spectrum-text enhanced learning and a time-centric gradient alignment mechanism. Key technical components—including spectral visualization, time-aware Mixture-of-Experts (MoE), gradient normalization, and reliability-aware dynamic weighting—enable VLT to significantly outperform existing methods across multiple industrial datasets. The model demonstrates exceptional robustness and generalization under complex conditions such as few-shot learning, noisy inputs, and missing modalities.

Abstract

Industrial time series serve as the foundation for Prognostics and Health Management (PHM) to ensure the reliability and safety of industrial equipment such as aero-engines. However, existing approaches are typically limited to single-modality modeling, which restricts their generalization in complex scenarios. Although recent advances in large language models (LLMs) provide new opportunities for multimodal learning, bridging continuous time-series signals and discrete textual semantics remains an open challenge. To this end, we propose VLT, a multimodal foundation model that jointly models time-series, frequency-spectrum visual representations, and textual knowledge. A key insight is to utilize the frequency spectrum as a visual bridge to connect continuous temporal signals with discrete semantics. Specifically, a Time-aware Mixture-of-Experts (Time-MoE) is designed to capture heterogeneous temporal dynamics, while a Frequency-Text Augmented Learner enables joint modeling of spectral and semantic features within a shared representation space. Furthermore, a time-centric gradient alignment mechanism is introduced to mitigate cross-modal optimization conflicts via gradient normalization and reliability-aware dynamic reweighting. Extensive experiments on multiple industrial datasets demonstrate that VLT outperforms state-of-the-art methods, achieving superior robustness and generalization under few-shot, noisy, and incomplete-modality settings.

Problem

• time series
• multimodal learning
• vision-language
• industrial intelligence
• semantic alignment

Innovation

• Vision-Language-Time Series
• Frequency Spectrum as Visual Bridge
• Time-aware Mixture-of-Experts
• Gradient Alignment
• Multimodal Foundation Model