NEST: Tackling Dataset-Level Distribution Shifts via Regime-Oriented Mixture-of-Experts

2026-07-07

AI Summary

This work addresses the performance degradation in long-term forecasting of complex systems caused by dataset-level distribution shifts arising from multiple operating regimes and dynamically evolving states. To tackle this challenge, the authors propose NEST, a novel framework that models structural changes through a two-stage dense mixture-of-experts architecture. NEST first performs unsupervised clustering in the moment–entropy space to identify distinct operating regimes, then employs a regime-aware routing mechanism coupled with geometric modulation to dynamically generate expert weights. Each expert functions as a specialized dynamic kernel that captures variable attention patterns specific to its corresponding regime. By explicitly modeling composite, dataset-level operating mechanisms—a capability absent in prior approaches—NEST achieves significant performance gains over existing methods on benchmarks spanning heterogeneous network traffic and physical phenomena.

Abstract

Accurate long-term forecasting in complex systems is frequently compromised by dataset-level distribution shifts, where diverse underlying behavioral modes and evolving system states drive the dynamic multivariate time-series. While existing methods predominantly focus on local temporal shifts, they fail to explicitly model the global structural challenge where datasets are composites of distinct operational regimes. In this paper, we propose NEST, a specialized framework designed to model and recompose these evolving structures through a two-phase dense MoE architecture. NEST first facilitates structural specialization by partitioning the dataset into distinct operational regimes through unsupervised clustering in a principled moment-entropy space. We introduce a regime-oriented router mechanism that generates initial expert weights based on temporal content, subsequently refined through geometric modulation to regime centroids. Crucially, rather than acting as monolithic predictors, individual experts function as specialized kernels that capture regime-specific dynamics by evolving unique variate-attention patterns. Extensive evaluations on diverse benchmarks, including heterogeneous network traffic and physical phenomena, demonstrate that NEST consistently achieves state-of-the-art performance. Our code and datasets are available at https://github.com/Aaralshin/NEST

Problem

• distribution shift
• long-term forecasting
• operational regimes
• multivariate time-series
• dataset-level shifts

Innovation

• distribution shift
• mixture-of-experts
• operational regimes
• moment-entropy space
• variate-attention