CaMeLoT: CaMeL orchestrated with Temporal logic for static verification and liveness

2026-09-16

AI Summary

为解决LLM代理执行计划的安全性问题，CaMeLoT通过将计划转换为有限状态转移系统并使用nuXmv模型检查器进行静态验证，确保在执行前符合安全策略。

Abstract

LLM-based agents generate and execute multi-step plans that invoke external tools which can access private data or execute commands. In this setting, security is a property of the entire execution that a plan creates, not just any single step. The plan itself is a critical artefact that captures the tool calls, control flow, and data dependencies. We present CaMeLoT, a complement to CaMeL, an existing defence against prompt injection in tool-using LLM agents. CaMeLoT extends CaMeL by adding a static verification layer that checks an agent's plan before any tool is invoked. CaMeLoT translates a generated plan into a finite-state transition system, labels it with tool calls, provenance and taint information, and checks it against temporal policies expressed in CTL using the nuXmv model checker. Because verification happens before execution, unsafe plans are rejected without using LLM calls or tool calls, saving tokens that runtime could have cost, as well as the need to unwind changes or teardown temporary sandboxes. When a verification fails, the model checker returns a counterexample to give feedback to the agent to repair the plan. We evaluate CaMeLoT on policies derived from the AgentDojo benchmark, SOC workflows, and prompt-extraction experiments, showing that it verifies a broad class of temporal properties before execution while preserving CaMeL's runtime-checkable coverage.

Problem

• static verification
• LLM-based agents
• security
• multi-step plans
• temporal logic

Innovation

• static verification
• temporal logic
• finite-state transition system
• CTL
• nuXmv model checker