LLM-Integrated Bayesian State Space Models for Multimodal Time-Series Forecasting

2025-10-23

AI Summary

Existing methods struggle to jointly model structured time series and unstructured text, while being constrained by fixed-length horizons and lacking uncertainty quantification capabilities. To address these limitations, we propose the LBS framework—the first unified architecture integrating large language models (LLMs) with Bayesian state-space models (SSMs). In LBS, SSMs capture latent temporal dynamics, whereas LLMs handle textual encoding/decoding and generate natural-language summaries. The framework supports variable-length input-output windows, principled Bayesian uncertainty quantification, and cross-modal joint forecasting. Evaluated on the TextTimeCorpus benchmark, LBS achieves a 13.20% improvement over state-of-the-art methods. Moreover, it generates high-fidelity, human-readable predictive explanations, significantly enhancing multimodal time-series generalization.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Large language models for search
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Forecasting in the real world requires integrating structured time-series data with unstructured textual information, but existing methods are architecturally limited by fixed input/output horizons and are unable to model or quantify uncertainty. We address this challenge by introducing LLM-integrated Bayesian State space models (LBS), a novel probabilistic framework for multimodal temporal forecasting. At a high level, LBS consists of two components: (1) a state space model (SSM) backbone that captures the temporal dynamics of latent states from which both numerical and textual observations are generated and (2) a pretrained large language model (LLM) that is adapted to encode textual inputs for posterior state estimation and decode textual forecasts consistent with the latent trajectory. This design enables flexible lookback and forecast windows, principled uncertainty quantification, and improved temporal generalization thanks to the well-suited inductive bias of SSMs toward modeling dynamical systems. Experiments on the TextTimeCorpus benchmark demonstrate that LBS improves the previous state-of-the-art by 13.20% while providing human-readable summaries of each forecast. Our work is the first to unify LLMs and SSMs for joint numerical and textual prediction, offering a novel foundation for multimodal temporal reasoning.

Problem

• Integrating structured time-series with unstructured text data
• Overcoming fixed input/output horizons in forecasting models
• Modeling and quantifying uncertainty in multimodal predictions

Innovation

• LLM-integrated Bayesian State Space Models for multimodal forecasting
• State space model captures latent dynamics for numerical and textual data
• Pretrained LLM encodes text inputs and decodes textual forecasts