Safe Meta-Reinforcement Learning via Information Space Reachability

2026-09-14

AI Summary

本文提出了一种安全的元强化学习框架，通过在信息空间中引入安全值函数来解决实际任务中的安全性问题，并开发了相应的算法以实现安全过滤和策略优化。

Abstract

Meta-reinforcement learning (meta-RL) enables agents to adapt to unseen tasks with limited experience. Despite its promise, the application of meta-RL in real-world tasks is hindered by safety requirements, which have been underexplored in prior work. In this paper, we propose a safe meta-RL framework that explicitly accounts for safety during adaptation. Our key insight is to reason about safety in the information space, which captures both the physical state and the agent's belief over the underlying task. Within this space, we introduce a safety value function that measures the probability of the agent avoiding unsafe regions indefinitely. We show that this function satisfies a self-consistency condition and a Bellman equation, which make it learnable via meta-RL. Based on this formulation, we develop a safe meta-RL algorithm that learns the safety value function and leverages it for safety filtering and constrained policy optimization. Experiments on meta-RL benchmarks demonstrate the effectiveness of the proposed method.

Problem

• Meta-Reinforcement Learning
• Safety
• Information Space

Innovation

• safe meta-RL
• information space
• safety value function
• self-consistency condition
• Bellman equation