Efficient Geothermal Well-Control Optimization via Diffusion-Surrogate Reinforcement Learning

2026-08-28

AI Summary

为解决增强型地热系统实时决策问题，提出了一种基于扩散-代理的强化学习框架，通过预测储层状态演化和经济回报，结合PPO算法进行高效策略训练。

Abstract

Real-time decision-making for enhanced geothermal systems (EGS) is challenging because long-term production periods involve high-dimensional control spaces and a large number of time-consuming high-fidelity hydrothermal simulations. Reinforcement learning provides a natural framework for state-dependent sequential control, but direct policy training with numerical simulators is computationally expensive. To address this issue, we propose a diffusion-surrogate guided reinforcement learning framework for long-horizon EGS well-control optimization. The reservoir temperature and pressure fields are used as system states, while injection rates are selected as control actions. A learned surrogate environment is constructed using conditional diffusion models to predict the evolution of reservoir temperature and pressure fields and a separate reward model to estimate the corresponding economic return. The surrogate environment is then integrated with Proximal Policy Optimization (PPO) for efficient policy training. Experiments on a fractured EGS benchmark show that the diffusion surrogate can accurately reproduce reservoir-state evolution over multiple control stages. The resulting surrogate-assisted PPO policy achieves competitive well-control performance compared with direct simulator-based PPO and existing optimization methods, while substantially reducing the dependence on expensive high-fidelity simulations. These results demonstrate the potential of diffusion-based surrogate environments for efficient reinforcement learning in geothermal well-control optimization.

Problem

• Enhanced Geothermal Systems
• Real-time Decision-making
• High-dimensional Control Spaces
• High-fidelity Hydrothermal Simulations
• Reinforcement Learning

Innovation

• Diffusion-Surrogate
• Reinforcement Learning
• Geothermal Well-Control
• Proximal Policy Optimization (PPO)
• Conditional Diffusion Models