End-to-end probabilistic hierarchical forecasting of large hierarchies via probabilistic top-down

2026-06-25

AI Summary

This work addresses the challenges of probabilistic forecasting in large-scale hierarchical time series, which often suffer from high computational costs, poor scalability, or reliance on strong modeling assumptions. The authors propose e2eTD, a novel end-to-end scalable probabilistic top-down approach that directly forecasts only the top-level smoothed series—approximately 0.3% of the total—and propagates predictions downward by modeling the joint distribution of historical disaggregation proportions. Bottom-level coherent probabilistic forecasts are then obtained by summing samples from the disaggregated paths, ensuring hierarchical consistency without requiring specialized hardware. Evaluated on the M5 and Favorita benchmarks, e2eTD achieves the lowest weighted quantile loss (ranking 11th out of 892 teams in the M5 competition) and demonstrates remarkable efficiency, processing hierarchies with 40,000 and 300,000 series in just 5 and 20 minutes, respectively, on a standard laptop.

Abstract

Retail and supply chain operations rely on demand forecasts to drive decisions, from replenishment at the product level to capacity planning at the store level. These forecasts should be probabilistic, to allow risk-aware decisions, and coherent across the aggregation hierarchy, so that decisions taken at different levels are not based on conflicting demand forecasts. However, producing coherent probabilistic forecasts is computationally demanding; at retail scale, with hierarchies of thousands of time series, this cost becomes a first-order operational concern. Existing two-step forecast-then-reconcile procedures and end-to-end neural models scale poorly, rely on restrictive assumptions, or require specialized hardware and engineering effort. We propose e2eTD, a fast and scalable method for probabilistic coherent forecasting of large hierarchical and grouped time series. e2eTD directly forecasts only a small subset of aggregate series (about 0.3\% of the hierarchy in our experiments), which are smoother and thus more predictable than the intermittent bottom series. The resulting forecast samples are propagated to the bottom level through a novel probabilistic top-down sampling algorithm, in which the historical disaggregation proportions are modeled as joint distributions, estimated in-sample. Coherent forecasts for all aggregation levels are then obtained by summing the joint bottom-level samples. On the two largest publicly available retail datasets, M5 and Favorita, e2eTD achieves the lowest weighted scaled pinball loss (the M5 competition's probabilistic score) across aggregation levels among all competing methods; it would have ranked 11th of 892 teams in the M5 Uncertainty competition. On a standard laptop, e2eTD runs in about five minutes on M5 ($\sim$40K series) and twenty minutes on Favorita ($\sim$300K series).

Problem

• probabilistic forecasting
• hierarchical forecasting
• forecast coherence
• large-scale time series
• retail demand forecasting

Innovation

• probabilistic hierarchical forecasting
• top-down reconciliation
• coherent forecasts
• scalable time series
• joint disaggregation distribution