A Bellman Optimality Equation for Plasticity

2026-09-09

AI Summary

本文针对持续强化学习中稳定性-可塑性权衡问题，通过定义新的可塑性和赋能概念，并提出了一种类似于赋能优化的贝尔曼最优方程来优化可塑性。

Abstract

In continual reinforcement learning, carefully managing the stability-plasticity tradeoff remains a core challenge. Recent work by Abel et al. (2025) formalized this dilemma by defining plasticity as the generalized directed information from an agent's observations to its actions, and empowerment as the generalized directed information from its actions to its observations. This formulation successfully reframes the traditional stability-plasticity tradeoff as an empowerment-plasticity tradeoff. However, while extensive literature exists on optimizing for empowerment, there is currently no research addressing the optimization of plasticity under this new definition. This paper presents preliminary work toward optimizing plasticity within Markov decision processes. We show that there exists a Bellman optimality equation for optimizing plasticity similar to previous work for empowerment.

Problem

• continual reinforcement learning
• stability-plasticity tradeoff
• plasticity optimization

Innovation

• Bellman optimality equation
• plasticity
• empowerment-plasticity tradeoff