ASPI: Seeking Ambiguity Clarification Amplifies Prompt Injection Vulnerability in LLM Agents

2026-05-17

AI Summary

This study addresses a critical oversight in existing research: the potential security risks posed by clarification interactions in large language model (LLM) agents. For the first time, clarification behavior is formally modeled as a distinct agent state, and the ASPI benchmark is introduced to systematically evaluate the differential susceptibility to prompt injection attacks between execution and clarification states. Controlled experiments across 728 task-attack scenarios and ten state-of-the-art models reveal that the clarification state substantially expands the attack surface—e.g., attack success rates for o3 and Gemini-1.5-Flash surge from 1.8% and 2.2% to 34.0% and 35.7%, respectively. These findings demonstrate that conventional evaluations significantly underestimate the real-world risks of interactive agents when handling ambiguous tasks, as robustness under clearly specified tasks does not generalize to scenarios requiring clarification.

Abstract

Clarification-seeking behavior is widely regarded as a desirable property of LLM agents, enabling them to resolve ambiguity before acting on underspecified tasks. However, the security implications of this interaction pattern remain unexplored. We investigate whether the transition from standard execution to a clarification-seeking state increases an agent's susceptibility to prompt injection attacks. We introduce ASPI (Ambiguous-State Prompt Injection), a benchmark of 728 task-attack scenarios that isolates clarification as a distinct agent state and measures how this state transition affects vulnerability under controlled conditions. Each benchmark instance is evaluated under matched execution and clarification settings: in the execution setting, the agent acts on a fully specified instruction and encounters adversarial content only through tool-returned data; in the clarification setting, the agent must first request and incorporate additional user input before acting. We evaluate ten frontier LLMs and find that clarification-seeking consistently and substantially amplifies vulnerability. For instance, attack success rises from 1.8% to 34.0% for o3 and from 2.2% to 35.7% for Gemini-3-Flash. A decomposition analysis reveals that this gap reflects both a state-dependent shift in how models process incoming content and a channel-specific effect arising from the agent-solicited clarification interface. These findings demonstrate that standard execution-time security evaluation systematically underestimates the attack surface of interactive agents, and that robustness under fully specified tasks does not translate to robustness under ambiguity. For reproducibility, our data and source code are available at https://github.com/scaleapi/aspi.

Problem

• prompt injection
• clarification-seeking
• LLM agents
• ambiguity
• security vulnerability

Innovation

• prompt injection
• clarification-seeking
• LLM agents
• security vulnerability
• ASPI benchmark