🤖 AI Summary
Existing tool-calling security mechanisms struggle to balance security and functionality in mixed-trust workflows, often resulting in false positives or false negatives. This work proposes PACT, a novel system that reframes agent security as a permission-binding problem. PACT leverages semantic role labeling to assign role-based semantics to tool parameters and integrates cross-step data provenance with role-specific trust contracts to enable fine-grained runtime validation. Evaluation on AgentDojo across five large language models demonstrates that PACT achieves 100% security on the three strongest models while recovering 38.1%–46.4% of utility—substantially outperforming current baseline approaches.
📝 Abstract
Tool-using LLM agents must act on untrusted webpages, emails, files, and API outputs while issuing privileged tool calls. Existing defenses often mediate trust at the granularity of an entire tool invocation, forcing a brittle choice in mixed-trust workflows: allow external content to influence a call and risk hijacked destinations or commands, or quarantine the call and block benign retrieval-then-act behavior. The key observation behind this paper is that indirect prompt injection becomes dangerous not when untrusted content appears in context, but when it determines an authority-bearing argument. We present \textsc{PACT} (\emph{Provenance-Aware Capability Contracts}), a runtime monitor that assigns semantic roles to tool arguments, tracks value provenance across replanning steps, and checks whether each argument's origin satisfies its role-specific trust contract. Under oracle provenance, \textsc{PACT} achieves 100\% utility and 100\% security on mixed-trust diagnostic suites, while flat invocation-level monitors incur false positives or false negatives. In full AgentDojo deployments across five models, \textsc{PACT} reaches 100\% security on the three strongest models while recovering 38.1--46.4\% utility, 8--16 percentage points above CaMeL at the same security level. Ablations show that both semantic roles and cross-step provenance are necessary. \textsc{PACT} reframes agent security as authority binding, and isolates the remaining deployment bottleneck to provenance inference and contract synthesis.