Statistical and economic evaluation of forecasts in electricity markets: beyond RMSE and MAE

2025-11-17

AI Summary

This study addresses the economic value assessment of electricity price forecasting in power markets, where conventional statistical metrics (e.g., RMSE, MAE) fail to reflect actual arbitrage returns when battery energy storage systems (BESS) make day-ahead charge/discharge decisions. To bridge this gap, we propose a novel class of shape-aware evaluation metrics that quantify temporal alignment between forecasted and actual price curves—replacing scalar error measures. Leveraging 192 synthetic forecasts and seven extended statistical indicators, we empirically quantify correlations between each metric and dynamic BESS arbitrage profits. Results show that RMSE and MAE exhibit only weak correlation with economic returns (|r| < 0.3), whereas curve-alignment metrics achieve strong positive correlation (r > 0.85), substantially improving the economic validity of model selection. The proposed framework establishes an interpretable, optimization-friendly, economics-driven paradigm for evaluating price forecasts tailored to energy storage applications.

Technology Category

• Game Theory and Economic Paradigms: Auctions and Market-Based Systems
• Machine Learning: Calibration & Uncertainty Quantification
• Search and Optimization: Metareasoning and Metaheuristics

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success

Abstract

In recent years, a rapid development of forecasting methods has led to an increase in the accuracy of predictions. In the literature, forecasts are typically evaluated using metrics such as Root Mean Squared Error (RMSE) and Mean Absolute Error (MAE). While appropriate for statistical assessment, these measures do not adequately reflect the economic value of forecasts. This study addresses the decision-making problem faced by a battery energy storage system, which must determine optimal charging and discharging times based on day-ahead electricity price forecasts. To explore the relationship between forecast accuracy and economic value, we generate a pool of 192 forecasts. These are evaluated using seven statistical metrics that go beyond RMSE and MAE, capturing various characteristics of the predictions and associated errors. We calculate the dynamic correlation between the statistical measures and gained profits to reveal that both RMSE and MAE are only weakly correlated with revenue. In contrast, measures that assess the alignment between predicted and actual daily price curves have a stronger relationship with profitability and are thus more effective for selecting optimal forecasts.

Problem

• Evaluating economic value of electricity price forecasts beyond statistical accuracy
• Assessing forecast quality for battery storage charging/discharging decisions
• Identifying statistical metrics that correlate strongly with profit generation

Innovation

• Evaluates forecasts with seven statistical metrics
• Analyzes correlation between accuracy and economic profits
• Uses daily price curve alignment for forecast selection