Approximate Dynamic Programming for Degradation-aware Market Participation of Battery Energy Storage Systems: Bridging Market and Degradation Timescales

2026-03-22

AI Summary

This work addresses the computational challenge arising from the coupling of multiple timescales in battery energy storage systems participating in electricity markets, where market dynamics operate on seconds-to-minutes scales while battery degradation unfolds over months to years. To tackle this, the authors propose an approximate dynamic programming framework that reduces the state space to state of charge and state of health, and introduces a novel pseudo-time axis parameterized by state of health to decouple offline policy learning from online decision-making. By integrating value function approximation, coarse-grained backward induction, and a physics-informed degradation model, the approach enables real-time optimization under high-fidelity degradation modeling. Backtesting against historical market data demonstrates that the proposed strategy significantly outperforms several hyperparameter-optimized benchmark methods in balancing economic returns and battery lifetime.

Technology Category

• Planning, Routing, and Scheduling: Planning with Markov Models (MDPs, POMDPs)
• Search and Optimization: Learning to Search
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Economics, Online Markets and Human Computation: Social networks and social learning
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Energy management for devices in mobile Web and WoT environments

Abstract

We present an approximate dynamic programming framework for designing degradation-aware market participation policies for battery energy storage systems. The approach employs a tailored value function approximation that reduces the state space to state of charge and battery health, while performing dynamic programming along a pseudo-time axis encoded by state of health. This formulation enables an offline/online computation split that separates long-term degradation dynamics (months to years) from short-term market dynamics (seconds to minutes) -- a timescale mismatch that renders conventional predictive control and dynamic programming approaches computationally intractable. The main computational effort occurs offline, where the value function is approximated via coarse-grained backward induction along the health dimension. Online decisions then reduce to a real-time tractable one-step predictive control problem guided by the precomputed value function. This decoupling allows the integration of high-fidelity physics-informed degradation models without sacrificing real-time feasibility. Backtests on historical market data show that the resulting policy outperforms several benchmark strategies with optimized hyperparameters.

Problem

• battery energy storage
• degradation
• market participation
• timescale mismatch
• dynamic programming

Innovation

• Approximate Dynamic Programming
• Battery Degradation
• Timescale Decoupling
• Value Function Approximation
• Market Participation