Entropy Regularized Reinforcement Learning for Zero-Sum Stochastic Differential Games in a Regime-Switching Jump-Diffusion Process

2026-06-26

AI Summary

This study addresses the challenges of parameter misspecification and abrupt environmental structural changes in classical stochastic differential games by proposing a zero-sum stochastic differential game framework grounded in entropy regularization. The approach models optimal policies as action probability distributions conditioned on continuous states, discrete regime states, and model parameters, and for the first time integrates distributional control and entropy regularization into regime-switching jump-diffusion processes. Coupled Hamilton-Jacobi-Bellman-Isaacs equations are derived via dynamic programming, yielding semi-analytical solutions for both the value function and equilibrium strategies in the linear-quadratic case. For general settings, a cross-regime Actor-Critic algorithm is developed, and numerical experiments demonstrate the pronounced effects of the temperature parameter and regime switching on both policies and value functions.

Abstract

To address parameter misspecification and sudden structural environmental changes in conventional stochastic differential game (SDG) frameworks, this paper introduces a distributional control approach that characterizes optimal strategies as probability distributions over actions, conditioned on the continuous state, the discrete regime state, and parameters. This forms a reinforcement learning framework for entropy-regularized zero-sum stochastic differential games (ERRL-ZSSDGs) in a regime-switching jump-diffusion process. Using the dynamic programming principle (DPP), we derive the associated coupled systems of Hamilton-Jacobi-Bellman-Isaacs (HJBI) equations, from which equilibrium strategies are expressed via gradients of the value function. For linear-quadratic problems, semi-analytical solutions for both value function and equilibrium strategies are obtained by solving a system of coupled ordinary differential equations (ODEs). In more general settings, an Actor-Critic policy improvement algorithm is developed to approximate the value functions and equilibrium policies across different regimes. The method is applied to an investment game, and numerical examples illustrate the effect of the temperature parameter and regime transitions on optimal policies and values.

Problem

• parameter misspecification
• regime-switching
• jump-diffusion
• stochastic differential games
• structural changes

Innovation

• Entropy Regularization
• Regime-Switching Jump-Diffusion
• Zero-Sum Stochastic Differential Games
• Distributional Control
• Actor-Critic Algorithm