🤖 AI Summary
Deep reinforcement learning exhibits significantly degraded robustness under adversarial perturbations, and existing research suffers from fragmented implementations, inconsistent evaluation protocols, and poor reproducibility. This work proposes the first standardized, open-source benchmarking framework for adversarial reinforcement learning, which unifies abstract interfaces for policies, attacks, defenses, and robustness metrics, and seamlessly integrates with Stable-Baselines3 and Gymnasium. Supporting algorithms such as DQN, PPO, and SAC, the framework encompasses 192 attack–defense combinations. Systematic evaluations on LunarLander and Highway-v0 reveal substantial environment-dependent variations in agent robustness, demonstrate that certain defense methods can be detrimental, and consistently identify temporal smoothing as an effective strategy for enhancing robustness. The code is publicly released.
📝 Abstract
Deep Reinforcement Learning (DRL) has achieved significant success in robotics and autonomous systems, yet remains vulnerable to adversarial perturbations that can severely degrade performance. Research in adversarial reinforcement learning is often limited by fragmented implementations, inconsistent evaluation protocols, and poor reproducibility. To address these challenges, we present \textbf{RoAd-RL}, an open-source benchmarking framework that provides unified abstractions for policies, attacks, defenses, and robustness metrics, together with reproducible evaluation pipelines and seamless integration with Stable-Baselines3 and Gymnasium.
We evaluate DQN, PPO, and SAC agents in LunarLander and Highway-v0 under 192 attack-defense configurations. Results reveal substantial variations in robustness across environments and show that some commonly used defenses can be more detrimental than the attacks they aim to mitigate, while temporal smoothing consistently achieves strong performance. RoAd-RL establishes a standardized benchmark for adversarial reinforcement learning research and is publicly available at https://pypi.org/project/road-rl.