Using dynamic loss weighting to boost improvements in forecast stability

2024-09-26

arXiv.org

AI Summary

To address prediction instability induced by data updates in rolling-origin forecasting, this paper proposes a dynamic loss weighting framework that jointly optimizes stability and accuracy. We introduce adaptive multi-objective loss mechanisms—specifically Task-Aware Random Weighting—for the first time in time-series forecasting, overcoming the limitations of static weighting schemes. Building upon the N-BEATS architecture, we integrate GradNorm, uncertainty-based weighting, and an enhanced task-aware random weighting strategy to formulate a dual-objective composite loss function, evaluated under the rolling-origin paradigm. Experiments on multiple standard benchmarks demonstrate that our approach improves forecast stability by 18–35% while maintaining or marginally surpassing the point prediction accuracy (measured by MAPE and RMSE) of the original N-BEATS model.

Technology Category

• Reasoning under Uncertainty: Stochastic Optimization
• Machine Learning: Time-Series/Data Streams
• Search and Optimization: Metareasoning and Metaheuristics

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Rolling origin forecast instability refers to variability in forecasts for a specific period induced by updating the forecast when new data points become available. Recently, an extension to the N-BEATS model for univariate time series point forecasting was proposed to include forecast stability as an additional optimization objective, next to accuracy. It was shown that more stable forecasts can be obtained without harming accuracy by minimizing a composite loss function that contains both a forecast error and a forecast instability component, with a static hyperparameter to control the impact of stability. In this paper, we empirically investigate whether further improvements in stability can be obtained without compromising accuracy by applying dynamic loss weighting algorithms, which change the loss weights during training. We show that existing dynamic loss weighting methods can achieve this objective and provide insights into why this might be the case. Additionally, we propose an extension to the Random Weighting approach -- Task-Aware Random Weighting -- which also achieves this objective.

Problem

• Rolling Origin Prediction
• Model Updating
• Stability and Accuracy

Innovation

• Dynamic Loss Weighting
• Stability Enhancement
• Task-Aware Randomized Weighting