Multi-layer Stack Ensembles for Time Series Forecasting

2025-11-19

AI Summary

To address the low adoption and performance instability of ensemble methods—particularly stacking—in time-series forecasting, this paper proposes a multi-layer stacking framework that hierarchically fuses heterogeneous base models with meta-learners, mitigating the task-transfer volatility inherent in single-stack ensembles. The method systematically integrates 33 ensemble models, including a newly designed nonlinear combination mechanism. We conduct large-scale empirical evaluation across 50 real-world time-series datasets. Results demonstrate that the framework consistently outperforms conventional linear-weighted ensembling and other state-of-the-art approaches across diverse forecasting tasks, delivering stable and statistically significant accuracy improvements. This work establishes a scalable, robust paradigm for time-series ensemble learning, advancing both methodological design and practical deployment of stacked ensembles.

Technology Category

• Machine Learning: Ensemble Methods
• Search and Optimization: Metareasoning and Metaheuristics
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs

Abstract

Ensembling is a powerful technique for improving the accuracy of machine learning models, with methods like stacking achieving strong results in tabular tasks. In time series forecasting, however, ensemble methods remain underutilized, with simple linear combinations still considered state-of-the-art. In this paper, we systematically explore ensembling strategies for time series forecasting. We evaluate 33 ensemble models -- both existing and novel -- across 50 real-world datasets. Our results show that stacking consistently improves accuracy, though no single stacker performs best across all tasks. To address this, we propose a multi-layer stacking framework for time series forecasting, an approach that combines the strengths of different stacker models. We demonstrate that this method consistently provides superior accuracy across diverse forecasting scenarios. Our findings highlight the potential of stacking-based methods to improve AutoML systems for time series forecasting.

Problem

• Explores ensemble strategies to improve time series forecasting accuracy
• Addresses underutilization of advanced ensemble methods in time series forecasting
• Proposes multi-layer stacking framework for superior forecasting performance

Innovation

• Proposes multi-layer stacking framework for time series
• Combines strengths of different stacker models systematically
• Demonstrates superior accuracy across diverse forecasting scenarios