Sparse Diffusion Autoencoder for Test-time Adapting Prediction of Complex Systems

2025-05-23

AI Summary

Existing long-term forecasting methods for complex systems suffer from spatial-temporal structure loss in latent representations, hindering effective modeling of spatial interactions and dynamic evolution. To address this, we propose a test-time adaptive sparse coding framework featuring a novel dynamic graph topology update mechanism to capture emergent spatiotemporal patterns. Our approach integrates a codebook-driven sparse graph encoder, graph neural ordinary differential equations (GNN-ODE) for continuous-time dynamics modeling, and a diffusion-based decoder. Furthermore, we introduce a joint online re-encoding and autoregressive prediction paradigm. Evaluated across diverse complex systems, our method achieves a 49.99% average reduction in prediction error while operating at only 1% of the original spatial resolution. This significantly enhances long-horizon forecasting accuracy and cross-system generalization capability.

Technology Category

• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Machine Learning: Graph-based Machine Learning

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Predicting the behavior of complex systems is critical in many scientific and engineering domains, and hinges on the model's ability to capture their underlying dynamics. Existing methods encode the intrinsic dynamics of high-dimensional observations through latent representations and predict autoregressively. However, these latent representations lose the inherent spatial structure of spatiotemporal dynamics, leading to the predictor's inability to effectively model spatial interactions and neglect emerging dynamics during long-term prediction. In this work, we propose SparseDiff, introducing a test-time adaptation strategy to dynamically update the encoding scheme to accommodate emergent spatiotemporal structures during the long-term evolution of the system. Specifically, we first design a codebook-based sparse encoder, which coarsens the continuous spatial domain into a sparse graph topology. Then, we employ a graph neural ordinary differential equation to model the dynamics and guide a diffusion decoder for reconstruction. SparseDiff autoregressively predicts the spatiotemporal evolution and adjust the sparse topological structure to adapt to emergent spatiotemporal patterns by adaptive re-encoding. Extensive evaluations on representative systems demonstrate that SparseDiff achieves an average prediction error reduction of 49.99% compared to baselines, requiring only 1% of the spatial resolution.

Problem

• Capturing spatial structure in spatiotemporal dynamics prediction
• Adapting to emergent dynamics during long-term system evolution
• Reducing prediction error while maintaining low spatial resolution

Innovation

• Test-time adaptation strategy for dynamic updates
• Codebook-based sparse encoder for graph topology
• Graph neural ODE with diffusion decoder