RoboJailBench: Benchmarking Adversarial Attacks and Defenses in Embodied Robotic Agents

2026-05-19

AI Summary

Existing evaluation frameworks lack a systematic assessment of the trade-off between safety and task utility for embodied agents under jailbreak attacks, and no standardized adversarial risk evaluation methodology exists. To address this gap, this work proposes RoboJailBench—the first standardized benchmark for jailbreak attacks targeting embodied AI. Built upon ISO standards and real-world incidents, it introduces an 18-category safety consequence taxonomy, constructs an intent-contrast dataset encompassing both adversarial and benign instructions, and integrates four attack strategies, two defense mechanisms, and multidimensional evaluation metrics. The benchmark extends five existing datasets and enables comprehensive evaluation across mainstream embodied vision-language models. Code and a public leaderboard are released to support community-wide benchmarking and advancement.

Abstract

Recent advances in Vision-Language Models (VLMs) facilitate a new class of embodied AI systems, where these models are integrated into physical platforms, e.g. robots and autonomous vehicles, to interpret visual scenes and execute natural language commands in diverse environments. Previous research has introduced jailbreak attacks and defenses for embodied AI. Their evaluations, however, rely on ad-hoc datasets, limited metrics, and emphasize attack success while neglecting the trade-off between security and the ability to follow benign commands. Existing benchmarks and evaluation frameworks either target traditional chat-based models or focus on non-adversarial safety evaluation for embodied AI; neither captures the adversarial risks, inputs, consequences, and evaluation criteria necessary for jailbreak attacks in embodied AI systems. In this paper, we address this gap with RoboJailBench, which consists of three core components. We establish a security taxonomy derived from ISO standards, regulatory rules, and documented incidents. This effort yields 18 categories of security violation consequences for embodied AI. We introduce an intent contrast dataset pipeline that augments existing datasets with paired adversarial and benign goals to measure both security and utility. Lastly, we provide an evolving repository with standardized metrics and a unified process for assessing and integrating new attacks and defenses. With this benchmark, we construct a new taxonomy-balanced dataset and augment five existing datasets. We integrate four attacks and two defenses to evaluate their performance on leading embodied VLMs. This benchmark provides the first standardized evaluation framework for jailbreak attacks in embodied AI and supports future research. We release our code, datasets, and artifacts, and maintain a leaderboard at https://purseclab.github.io/benchmark-for-robotics-security.

Problem

• embodied AI
• jailbreak attacks
• adversarial robustness
• benchmarking
• security evaluation

Innovation

• embodied AI
• jailbreak attacks
• Vision-Language Models
• adversarial benchmarking
• security-utility trade-off