Marginalization Consistent Probabilistic Forecasting of Irregular Time Series via Mixture of Separable flows

2024-06-11

AI Summary

Existing probabilistic forecasting models for irregular time series with missing values—e.g., ProFITi—suffer from marginalization inconsistency: marginalizing their joint predictive distribution over subsets of variables yields results inconsistent with direct predictions of those subsets, leading to inaccurate marginal forecasts. Method: We propose the first analytically marginalizable streaming probabilistic forecasting framework, integrating mixture-of-experts conditional normalizing flows with implicit multivariate Gaussian processes to guarantee exact consistency between joint predictions and all-order marginal predictions over arbitrary variable subsets. Contribution/Results: On four benchmark datasets, our method achieves significantly superior marginal forecast accuracy compared to all marginally consistent baselines, while attaining joint prediction performance competitive with ProFITi. Crucially, it establishes fundamental marginal consistency—resolving a long-standing modeling bottleneck in probabilistic forecasting for irregular, incomplete time series.

Technology Category

• Reasoning under Uncertainty: Relational Probabilistic Models
• Machine Learning: Time-Series/Data Streams
• Data Mining & Knowledge Management: Data Stream Mining

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Data management and stream processing for Web, mobile and wireless applications
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Probabilistic forecasting models for joint distributions of targets in irregular time series with missing values are a heavily under-researched area in machine learning, with, to the best of our knowledge, only two Models have been researched so far: The Gaussian Process Regression model, and ProFITi. While ProFITi, thanks to using multivariate normalizing flows, is very expressive, leading to better predictive performance, it suffers from marginalization inconsistency: It does not guarantee that the marginal distributions of a subset of variables in its predictive distributions coincide with the directly predicted distributions of these variables. When asked to directly predict marginal distributions, they are often vastly inaccurate. We propose MOSES (Marginalization Consistent Mixture of Separable Flows), a model that parametrizes a stochastic process through a mixture of several latent multivariate Gaussian Processes combined with separable univariate Normalizing Flows. In particular, MOSES can be analytically marginalized, allowing it to directly answer a wider range of probabilistic queries than most competitors. Experiments on four datasets show that MOSES achieves both accurate joint and marginal predictions, surpassing all other marginalization consistent baselines, while only trailing slightly behind ProFITi in joint prediction, but vastly superior when predicting marginal distributions.

Problem

• Develops marginalization-consistent probabilistic forecasting for irregular time series
• Addresses inaccuracy in marginal distribution predictions of existing models
• Proposes MOSES for accurate joint and marginal distribution predictions

Innovation

• Uses mixture of separable flows
• Analytically marginalizes stochastic processes
• Combines Gaussian Processes with Normalizing Flows