Natural-gas storage modelling by deep reinforcement learning

2025-11-04

Proceedings of the 6th ACM International Conference on AI in Finance

AI Summary

This study investigates how natural gas storage management affects market equilibrium and resilience to supply shocks. We propose GasRL, a novel simulation framework that integrates deep reinforcement learning—specifically the Soft Actor-Critic algorithm—into a market equilibrium model for joint optimization of storage operations, price stability, supply-demand clearing, and operator profitability. Unlike conventional approaches reliant on historical price data for calibration, GasRL generates realistic, seasonally varying equilibrium price dynamics with inherent volatility without parameter tuning. Our method overcomes the limitations of traditional calibrated models by enabling multi-objective, end-to-end optimization. Results demonstrate that optimal storage policies significantly dampen price volatility and enhance market resilience; furthermore, EU-mandated minimum storage thresholds substantially improve system robustness against abrupt supply disruptions. This work establishes an interpretable, scalable, AI-driven paradigm for energy policy design and intelligent gas infrastructure decision-making.

Technology Category

• Search and Optimization: Learning to Search
• Game Theory and Economic Paradigms: Equilibrium
• Intelligent Robots: Learning & Optimization for ROB

Application Category

• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

We introduce GasRL, a simulator that couples a calibrated representation of the natural gas market with a model of storage-operator policies trained with deep reinforcement learning (RL). We use it to analyse how optimal stockpile management affects equilibrium prices and the dynamics of demand and supply. We test various RL algorithms and find that Soft Actor Critic (SAC) exhibits superior performance in the GasRL environment: multiple objectives of storage operators - including profitability, robust market clearing and price stabilisation - are successfully achieved. Moreover, the equilibrium price dynamics induced by SAC-derived optimal policies have characteristics, such as volatility and seasonality, that closely match those of real-world prices. Remarkably, this adherence to the historical distribution of prices is obtained without explicitly calibrating the model to price data. We show how the simulator can be used to assess the effects of EU-mandated minimum storage thresholds. We find that such thresholds have a positive effect on market resilience against unanticipated shifts in the distribution of supply shocks. For example, with unusually large shocks, market disruptions are averted more often if a threshold is in place.

Problem

• Optimizing natural gas storage management using reinforcement learning
• Analyzing storage policies' impact on equilibrium prices and supply
• Evaluating EU storage mandates' effect on market shock resilience

Innovation

• Deep reinforcement learning optimizes gas storage policies
• Soft Actor Critic algorithm achieves multiple storage objectives
• Simulator analyzes storage thresholds impact without price calibration