ceLLMate: Sandboxing Browser AI Agents

2025-12-14

AI Summary

Browser-based intelligent agents (BUAs) are vulnerable to prompt injection attacks, leading to privacy breaches or unauthorized state modifications. To address this, we propose the first browser-level sandboxing framework that enforces fine-grained permission control via semantic mapping and policy prediction. Our key contributions are: (1) the agent sitemap—a novel semantic abstraction bridging UI interactions and declarative security policies; (2) a dual-driven policy generation mechanism leveraging both website-provided declarations and natural-language task specifications; and (3) a lightweight, extension-based implementation integrating UI behavior abstraction, NL task parsing, mandatory policy enforcement, and runtime policy injection. Evaluated in realistic settings, our framework effectively mitigates diverse prompt injection attacks with negligible runtime overhead (<0.5% CPU increase), requires no modification to underlying LLMs or agents, and supports arbitrary LLM-powered BUAs transparently.

Technology Category

• Application Domains: Security
• Natural Language Processing: Prompt Engineering / Prompting
• Multiagent Systems: Adversarial Agents

Application Category

• Responsible Web: Consent frameworks and practices on the web
• Security and Privacy: Browser and end-point security
• Search and Retrieval-Augmented AI: Agentic search

Abstract

Browser-using agents (BUAs) are an emerging class of autonomous agents that interact with web browsers in human-like ways, including clicking, scrolling, filling forms, and navigating across pages. While these agents help automate repetitive online tasks, they are vulnerable to prompt injection attacks that can trick an agent into performing undesired actions, such as leaking private information or issuing state-changing requests. We propose ceLLMate, a browser-level sandboxing framework that restricts the agent's ambient authority and reduces the blast radius of prompt injections. We address two fundamental challenges: (1) The semantic gap challenge in policy enforcement arises because the agent operates through low-level UI observations and manipulations; however, writing and enforcing policies directly over UI-level events is brittle and error-prone. To address this challenge, we introduce an agent sitemap that maps low-level browser behaviors to high-level semantic actions. (2) Policy prediction in BUAs is the norm rather than the exception. BUAs have no app developer to pre-declare sandboxing policies, and thus, ceLLMate pairs website-authored mandatory policies with an automated policy-prediction layer that adapts and instantiates these policies from the user's natural-language task. We implement ceLLMate as an agent-agnostic browser extension and demonstrate how it enables sandboxing policies that effectively block various types of prompt injection attacks with negligible overhead.

Problem

• Sandboxing browser AI agents against prompt injection attacks
• Bridging semantic gap between low-level UI events and high-level actions
• Automating policy prediction for agents without pre-declared policies

Innovation

• Browser-level sandboxing framework restricts agent authority
• Agent sitemap maps low-level behaviors to semantic actions
• Automated policy prediction adapts policies from user tasks