ResSafe: Learning Safety Filtering with Residual Reinforcement Learning for Humanoids

2026-09-14

AI Summary

本文针对人形机器人安全控制问题，提出使用残差强化学习方法作为隐式安全过滤机制，将性能与安全解耦以提高安全性。

Abstract

Safe control of humanoid robots remains challenging due to their high-dimensional dynamics, contact-rich interactions, and sensitivity to disturbances. Although reinforcement learning has enabled effective locomotion and motion tracking, learned policies can still generate unsafe actions that lead to instability or falls. In this work, we propose residual reinforcement learning as an implicit safety-filtering mechanism for safe humanoid control. Instead of relying on a single nominal policy to simultaneously balance performance, safety, and robustness, we decouple performance and safety. The nominal policy focuses solely on task performance, while a residual policy learns safety corrections. This decoupling leads to a better performance--safety Pareto trade-off and avoids the need for careful tuning of multiple competing reward terms within a single policy training. We show that the residual policy can act as an implicit safety filter.

Problem

• humanoid robots
• safety
• reinforcement learning
• high-dimensional dynamics
• contact-rich interactions

Innovation

• Residual Reinforcement Learning
• Safety Filtering
• Humanoid Robots