CmdNeedle: Measuring the Incompleteness of Command Denylists for AI Agents

2026-06-13

AI Summary

This work addresses the widespread reliance of on-device AI agents on command denylists to restrict hazardous operations, revealing that such mechanisms are fundamentally vulnerable due to inherent incompleteness. The study presents the first systematic modeling and empirical analysis of denylist gaps, introducing CmdNeedle—an automated detection framework that leverages large language models to generate potential bypass commands, validates them through sandboxed execution, and iteratively refines the denylist via feedback-driven repair. Evaluated on 1,709 real-world denylists encompassing 13,332 rules, the framework demonstrates that 69.0%–98.6% of denylists contain exploitable bypass vulnerabilities. These findings underscore the pervasiveness and root causes of such weaknesses and offer a scalable pathway for detecting and mitigating security flaws in AI agent command filtering systems.

Abstract

The adoption of AI agents is increasing rapidly. Terminal AI agents, i.e., AI agents that run in terminal environments, are a widely used type of AI agents. Terminal AI agents rely heavily on shell command execution to interact with the host systems. They adopt a three-list command-gating mechanism to mitigate security risks introduced by command execution, with denylists serving as the load-bearing component. However, modern operating systems often ship a large, ever-expanding set of shell commands with complex functionalities. Our observation is that even a built-in denylist of Claude Code, well-maintained by its developers, can overlook bypass commands that invalidate its effectiveness. Such negligence leads to fragile command denylists that cannot even block operations that practitioners expect them to block. This paper presents the first systematic characterization of command denylist fragility in terminal AI agents. The paper formalizes the command denylist fragility problem and proposes an LLM-driven pipeline, CmdNeedle, to detect such fragility. It prompts the LLM to propose possible bypasses and iteratively repairs them using feedback from a validator that executes them in a sandbox. In the evaluation, we applied CmdNeedle to 1,709 real-world command denylists (containing 13,332 denylist rules) collected from GitHub. The evaluation shows several key findings, including that 69.0--98.6% of the denylists are fragile, that this fragility occurs consistently across projects and agents, and the validity of several possible root causes for this fragility. Our pipeline and findings will hopefully facilitate future research and practice regarding the command denylists used by AI agents.

Problem

• command denylist
• AI agents
• terminal security
• bypass commands
• denylist fragility

Innovation

• command denylist
• AI agent security
• LLM-driven testing
• bypass detection
• terminal sandboxing