Dynamical Diffusion: Learning Temporal Dynamics with Diffusion Models

2025-03-02

AI Summary

Existing diffusion models for time-series forecasting neglect intrinsic temporal dynamics, resulting in poor temporal coherence of generated sequences. To address this, we propose TimeDiffusion—a novel framework that explicitly incorporates time-dependent modeling into both the forward and reverse diffusion processes for the first time: the forward process employs temporal reparameterization to characterize state transitions, while the reverse process performs history-conditioned sampling to enable dynamic state evolution. Our approach integrates a time-aware diffusion architecture, a learnable state-transition mechanism, and a conditional reverse-sampling strategy. Extensive experiments on scientific spatiotemporal forecasting, video prediction, and multivariate time-series forecasting demonstrate that TimeDiffusion significantly improves temporal consistency and predictive accuracy, consistently outperforming state-of-the-art conditional diffusion baselines across all benchmarks.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Time-Series/Data Streams
• Planning, Routing, and Scheduling: Temporal Planning

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Diffusion models have emerged as powerful generative frameworks by progressively adding noise to data through a forward process and then reversing this process to generate realistic samples. While these models have achieved strong performance across various tasks and modalities, their application to temporal predictive learning remains underexplored. Existing approaches treat predictive learning as a conditional generation problem, but often fail to fully exploit the temporal dynamics inherent in the data, leading to challenges in generating temporally coherent sequences. To address this, we introduce Dynamical Diffusion (DyDiff), a theoretically sound framework that incorporates temporally aware forward and reverse processes. Dynamical Diffusion explicitly models temporal transitions at each diffusion step, establishing dependencies on preceding states to better capture temporal dynamics. Through the reparameterization trick, Dynamical Diffusion achieves efficient training and inference similar to any standard diffusion model. Extensive experiments across scientific spatiotemporal forecasting, video prediction, and time series forecasting demonstrate that Dynamical Diffusion consistently improves performance in temporal predictive tasks, filling a crucial gap in existing methodologies. Code is available at this repository: https://github.com/thuml/dynamical-diffusion.

Problem

• Addresses underutilized temporal dynamics in diffusion models.
• Introduces Dynamical Diffusion for coherent sequence generation.
• Improves performance in temporal predictive learning tasks.

Innovation

• Incorporates temporally aware forward and reverse processes
• Models temporal transitions at each diffusion step
• Achieves efficient training via reparameterization trick