Robust Gymnasium: A Unified Modular Benchmark for Robust Reinforcement Learning

2025-02-27

AI Summary

Current robust reinforcement learning (Robust RL) lacks standardized, systematic benchmarks; existing evaluations are typically confined to isolated environments and single perturbation types, hindering algorithmic comparability and generalization. Method: We introduce the first unified, modular benchmark for Robust RL—covering state, reward, action, and environment perturbations—implemented across 60+ reproducible tasks in control, robotics, safety-critical domains, and multi-agent settings. Built on the Gymnasium ecosystem, it features a novel, fully composable perturbation modeling framework supporting stochastic/adversarial perturbations, delays, noise injection, and customizable perturbation combinations and evaluation protocols. Contribution/Results: Extensive experiments expose widespread fragility of mainstream RL and Robust RL algorithms under cross-perturbation generalization, establishing a new quantitative standard for robustness evaluation and enabling principled, large-scale assessment of algorithmic resilience.

Technology Category

• Machine Learning: Adversarial Learning & Robustness
• Natural Language Processing: Safety and Robustness
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Driven by inherent uncertainty and the sim-to-real gap, robust reinforcement learning (RL) seeks to improve resilience against the complexity and variability in agent-environment sequential interactions. Despite the existence of a large number of RL benchmarks, there is a lack of standardized benchmarks for robust RL. Current robust RL policies often focus on a specific type of uncertainty and are evaluated in distinct, one-off environments. In this work, we introduce Robust-Gymnasium, a unified modular benchmark designed for robust RL that supports a wide variety of disruptions across all key RL components-agents' observed state and reward, agents' actions, and the environment. Offering over sixty diverse task environments spanning control and robotics, safe RL, and multi-agent RL, it provides an open-source and user-friendly tool for the community to assess current methods and foster the development of robust RL algorithms. In addition, we benchmark existing standard and robust RL algorithms within this framework, uncovering significant deficiencies in each and offering new insights.

Problem

• Standardized benchmarks for robust RL
• Unified modular evaluation framework
• Assessing resilience in diverse RL disruptions

Innovation

• Unified modular robust RL benchmark
• Supports diverse disruptions comprehensively
• Open-source tool for algorithm assessment