Distribution-Aware Diffusion-LLM for Robust Ultra-Long-Term Time Series Forecasting

2026-06-22

AI Summary

This work addresses the challenges of inadequate probabilistic calibration and difficulties in aligning heterogeneous representations when applying large language models (LLMs) to multivariate time series forecasting. To overcome these limitations, we propose a novel framework that integrates conditional diffusion mechanisms with LLMs. Our approach jointly models the conditional distribution of future data within a shared latent space, enabling semantic alignment and distribution-aware prediction, while incorporating distributional regularization to enhance robustness. As the first study to combine distribution-aware diffusion processes with LLMs for time series forecasting, our method achieves significant performance gains over existing approaches across six long-horizon benchmarks—including ETT, Weather, and ECL—with particularly strong results in ultra-long-term and few-shot forecasting scenarios.

Abstract

Time series forecasting is a fundamental machine learning task. Recent work has explored Large Language Models (LLMs) for this purpose due to their strong generalization, pattern recognition, and zero-shot or few-shot capabilities. Despite their suitability for long-context learning, LLMs face challenges in multimodal settings: they lack calibrated probabilistic modeling for non-text data and struggle to align heterogeneous representations. To address these issues, we propose a new framework Diffusion-LLM that integrates a conditional diffusion model into an LLM-based forecasting pipeline. This joint design enables learning the conditional distribution of future data while improving semantic alignment in a shared latent space. We evaluate Diffusion-LLM on six long-term forecasting benchmarks, including ETT, Weather, and ECL. Our method consistently outperforms existing LLM-based baseline, achieving notable gains in ultra-long-term and few-shot forecasting and demonstrating the value of distribution-aware regularization for enhancing robustness and generalization in time series LLMs.

Problem

• time series forecasting
• Large Language Models
• probabilistic modeling
• heterogeneous representations
• multimodal settings

Innovation

• Diffusion-LLM
• conditional diffusion model
• distribution-aware forecasting
• ultra-long-term time series
• LLM-based time series forecasting