🤖 AI Summary
To address multi-agent strategic interactions and price volatility arising from distributed energy resource (DER) aggregators participating in wholesale electricity markets, this paper proposes a hybrid mean-field framework: inter-aggregator competition is modeled as a mean-field game (MFG), while aggregator-level DER dispatch is formulated as mean-field control (MFC), integrated with deep reinforcement learning for adaptive policy optimization. This framework uniquely unifies the dynamic co-evolution of repeated bidding behaviors among multiple aggregators and their collective impact on nodal marginal prices (LMPs). Simulation results demonstrate rapid LMP convergence to a steady state and significant suppression of price volatility through synergistic coordination between battery storage response and mean-field learning. The approach simultaneously enhances market stability and resource allocation efficiency, offering a scalable theoretical and practical paradigm for market mechanism design under high DER penetration.
📝 Abstract
The integration of distributed energy resources (DERs) into wholesale energy markets can greatly enhance grid flexibility, improve market efficiency, and contribute to a more sustainable energy future. As DERs -- such as solar PV panels and energy storage -- proliferate, effective mechanisms are needed to ensure that small prosumers can participate meaningfully in these markets. We study a wholesale market model featuring multiple DER aggregators, each controlling a portfolio of DER resources and bidding into the market on behalf of the DER asset owners. The key of our approach lies in recognizing the repeated nature of market interactions the ability of participants to learn and adapt over time. Specifically, Aggregators repeatedly interact with each other and with other suppliers in the wholesale market, collectively shaping wholesale electricity prices (aka the locational marginal prices (LMPs)). We model this multi-agent interaction using a mean-field game (MFG), which uses market information -- reflecting the average behavior of market participants -- to enable each aggregator to predict long-term LMP trends and make informed decisions. For each aggregator, because they control the DERs within their portfolio under certain contract structures, we employ a mean-field control (MFC) approach (as opposed to a MFG) to learn an optimal policy that maximizes the total rewards of the DERs under their management. We also propose a reinforcement learning (RL)-based method to help each agent learn optimal strategies within the MFG framework, enhancing their ability to adapt to market conditions and uncertainties. Numerical simulations show that LMPs quickly reach a steady state in the hybrid mean-field approach. Furthermore, our results demonstrate that the combination of energy storage and mean-field learning significantly reduces price volatility compared to scenarios without storage.