Hidden in Plain Sight: Benchmarking Agent Safety Against Decomposition Attacks with DECOMPBENCH

2026-06-11

AI Summary

This work addresses the vulnerability of large language model (LLM) agents to decomposition attacks, wherein harmful tasks are strategically broken down into seemingly benign subtasks that bypass existing safety mechanisms. The authors introduce DeCompBench, the first systematic evaluation benchmark specifically designed for such attacks, grounded in the principle of “decomposition-as-design.” It models realistic and feasible decomposition pathways using graph structures and incorporates customized decomposers, multi-tool calling, and multi-turn interactive scenarios to rigorously assess agent safety. Experimental results reveal that while LLM agents consistently refuse to execute overtly harmful tasks as a whole, their refusal rates drop significantly when confronted with decomposed subtasks, often inadvertently aiding the completion of malicious objectives—thereby exposing critical gaps in current safety protocols.

Abstract

LLM-based Agents are becoming increasingly capable and widely deployed, creating growing incentives for adversarial misuse in the real-world. A key emerging threat is Decomposition Attacks \cite{glukhov2024breach, jones2024adversaries} in which a harmful task is broken into simpler, benign subtasks that evade safety mechanisms when executed separately but cumulatively fulfill the malicious intent. Although recent benchmarks assess agent safety in multi-turn and multi-tool-use settings, they do not explicitly capture this form of decompositional misuse and may not represent realistic adversarial execution flows. To this end, we introduce DeCompBench, a benchmark designed specifically to evaluate agentic safety under decomposition attacks. DeCompBench is created with a decomposition-by-design principle using a graphical framework and enables harmful task decomposition into individually benign and executable subtasks with realistic workflows. Our experiments using a custom decomposer show that state-of-the-art agents exhibit high refusal rates on monolithic harmful tasks, but significantly lower refusal rates on their decomposed variants, while often inadvertently fulfilling the adversarial objectives. These findings underscore the need for safety evaluations against decomposition attacks and corresponding defenses. Our dataset is publicly available and can be found at https://huggingface.co/datasets/decompositionbench/DeCompBench.

Problem

• Decomposition Attacks
• Agent Safety
• Adversarial Misuse
• Safety Benchmarking
• LLM-based Agents

Innovation

• Decomposition Attacks
• Agent Safety
• DeCompBench
• Adversarial Misuse
• Safety Evaluation