Online Multivariate Regularized Distributional Regression for High-dimensional Probabilistic Electricity Price Forecasting

2025-04-03

AI Summary

Real-time decision-making in electricity markets demands short-term probabilistic price forecasting that simultaneously achieves high accuracy, computational efficiency, and interpretability. Method: We propose an online-updatable high-dimensional multivariate distributional regression model. It integrates multivariate distribution regression with online coordinate-descent LASSO to jointly model the mean, variance, and dependence structure of 24-hour day-ahead prices. A novel regularization strategy—based on dependency-complexity paths—enables dynamic sparsity learning and early stopping. We further introduce adaptive Copula modeling for time-varying dependencies and incorporate real-time marginal distribution estimation. Contribution/Results: Evaluated on the German day-ahead market, our model significantly outperforms baselines—including online univariate + static Copula and online LASSO-ARX—in both calibration and sharpness. Training speed improves by 80–400×, while maintaining high predictive accuracy and strong interpretability through sparse, physically meaningful feature selection.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Relational Probabilistic Models
• Planning, Routing, and Scheduling: Model-Based Reasoning

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Probabilistic electricity price forecasting (PEPF) is a key task for market participants in short-term electricity markets. The increasing availability of high-frequency data and the need for real-time decision-making in energy markets require online estimation methods for efficient model updating. We present an online, multivariate, regularized distributional regression model, allowing for the modeling of all distribution parameters conditional on explanatory variables. Our approach is based on the combination of the multivariate distributional regression and an efficient online learning algorithm based on online coordinate descent for LASSO-type regularization. Additionally, we propose to regularize the estimation along a path of increasingly complex dependence structures of the multivariate distribution, allowing for parsimonious estimation and early stopping. We validate our approach through one of the first forecasting studies focusing on multivariate probabilistic forecasting in the German day-ahead electricity market while using only online estimation methods. We compare our approach to online LASSO-ARX-models with adaptive marginal distribution and to online univariate distributional models combined with an adaptive Copula. We show that the multivariate distributional regression, which allows modeling all distribution parameters - including the mean and the dependence structure - conditional on explanatory variables such as renewable in-feed or past prices provide superior forecasting performance compared to modeling of the marginals only and keeping a static/unconditional dependence structure. Additionally, online estimation yields a speed-up by a factor of 80 to over 400 times compared to batch fitting.

Problem

• Modeling multivariate dependence structures in electricity prices
• Enabling scalable high-dimensional probabilistic forecasting
• Balancing computational efficiency with predictive accuracy

Innovation

• Online algorithm for multivariate distributional regression models
• Combines coordinate descent with LASSO-type regularization
• Regularized estimation path with early stopping