A Transformer approach for Electricity Price Forecasting

2024-03-24

arXiv.org

AI Summary

Electricity price forecasting (EPF) demands accurate modeling of complex temporal dynamics, yet conventional approaches often rely on hybrid architectures combining Transformers with RNNs or CNNs. Method: We propose a pure, end-to-end Transformer model for EPF—eliminating all auxiliary structures—and leverage self-attention exclusively to capture multiscale temporal dependencies and intrinsic periodicities. Contribution/Results: This work presents the first systematic empirical validation of a standalone Transformer’s sufficiency and superiority in EPF. We release an open-source, unified evaluation toolkit—including standardized datasets, benchmark models, and fully reproducible code—to enable fair, transparent comparisons. Extensive experiments across multiple real-world electricity markets demonstrate that our model significantly outperforms classical baselines (e.g., ARIMA, LSTM), achieving average MAE reductions of 12.7%–18.3%. Moreover, it exhibits enhanced robustness under volatile market conditions. The model thus provides high-accuracy, interpretable short-term forecasts critical for power system scheduling and market operations.

Technology Category

• Machine Learning: Mixture of Experts (MoE)
• Computer Vision: Large Vision Models
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Web Mining and Content Analysis: Large pretrained models with web data
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

This paper presents a novel approach to electricity price forecasting (EPF) using a pure Transformer model. As opposed to other alternatives, no other recurrent network is used in combination to the attention mechanism. Hence, showing that the attention layer is enough for capturing the temporal patterns. The paper also provides fair comparison of the models using the open-source EPF toolbox and provide the code to enhance reproducibility and transparency in EPF research. The results show that the Transformer model outperforms traditional methods, offering a promising solution for reliable and sustainable power system operation.

Problem

• Electricity price forecasting using pure Transformer model
• Capturing temporal patterns solely with attention mechanism
• Outperforming traditional methods for reliable power operation

Innovation

• Pure Transformer model for electricity forecasting
• Attention mechanism captures temporal patterns alone
• Outperforms traditional methods in power systems