MoGU: Mixture-of-Gaussians with Uncertainty-based Gating for Time Series Forecasting

2025-10-08

AI Summary

Traditional Mixture-of-Experts (MoE) models for time-series forecasting produce only point estimates and lack explicit uncertainty quantification. To address this, we propose MoGU—a Gaussian Mixture-based Uncertainty-aware MoE framework—where each expert outputs a Gaussian distribution (mean and variance), and the gating mechanism dynamically weights experts based on their predicted variances rather than relying solely on input-driven gating networks. This variance-aware gating enables error-sensitive uncertainty estimation. MoGU is end-to-end trainable and achieves state-of-the-art performance across multiple benchmark datasets, significantly outperforming both single-expert baselines and standard MoE models. It delivers high-accuracy point forecasts while yielding well-calibrated predictive uncertainty, thereby enhancing forecast reliability and decision-making robustness.

Technology Category

• Machine Learning: Mixture of Experts (MoE)
• Reasoning under Uncertainty: Relational Probabilistic Models
• Planning, Routing, and Scheduling: Scheduling under Uncertainty

Application Category

• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

We introduce Mixture-of-Gaussians with Uncertainty-based Gating (MoGU), a novel Mixture-of-Experts (MoE) framework designed for regression tasks and applied to time series forecasting. Unlike conventional MoEs that provide only point estimates, MoGU models each expert's output as a Gaussian distribution. This allows it to directly quantify both the forecast (the mean) and its inherent uncertainty (variance). MoGU's core innovation is its uncertainty-based gating mechanism, which replaces the traditional input-based gating network by using each expert's estimated variance to determine its contribution to the final prediction. Evaluated across diverse time series forecasting benchmarks, MoGU consistently outperforms single-expert models and traditional MoE setups. It also provides well-quantified, informative uncertainties that directly correlate with prediction errors, enhancing forecast reliability. Our code is available from: https://github.com/yolish/moe_unc_tsf

Problem

• Modeling expert outputs as Gaussian distributions for forecasts
• Using uncertainty-based gating to determine expert contributions
• Providing well-quantified uncertainties to enhance forecast reliability

Innovation

• Models expert outputs as Gaussian distributions
• Uses uncertainty-based gating instead of input-based
• Quantifies both forecast mean and variance directly