Minim: Privacy-Aware Minimal View for Agents via Trusted Local Sanitization

2026-06-11

AI Summary

This work addresses the privacy risks inherent in deploying large language model (LLM) agents within complex digital environments, where reliance on complete UI state for decision-making often leads to unintended disclosure of sensitive, task-irrelevant information such as CAPTCHAs or private notifications. To mitigate this, the authors propose a privacy-preserving framework grounded in contextual integrity theory, employing a trusted local proxy on the client side. This proxy implements a dual-scoring mechanism that evaluates both the sensitivity of UI elements and their necessity for the agent’s task, applying a ternary disclosure policy to minimally transform the interface—retaining essential content, abstracting sensitive attributes, or removing irrelevant elements. Evaluated on real-world UI data from WebArena, the approach significantly reduces sensitive data exposure while preserving task-critical semantics and interaction capabilities, effectively balancing privacy protection with agent performance.

Abstract

Modern LLM-powered autonomous agents increasingly rely on rich user interface (UI) state observations to achieve reliable action grounding in complex digital environments. However, many deployments transmit the full UI state to remote inference servers even when most elements are irrelevant to the current task, which can leak sensitive but unnecessary context such as authentication codes, private notifications, and background application states. We propose MINIM, a trusted local broker that performs privacy-aware minimization on the client side before any observation leaves the device. Grounded in Contextual Integrity (CI), MINIM learns a dual-score representation for each UI element by predicting an inherent sensitivity score (s) and a task-conditioned necessity score (n). These scores drive a ternary disclosure policy that keeps essential elements, abstracts sensitive attributes when needed, and removes task-irrelevant content. We optimize a CI-aware objective that penalizes necessity errors more strongly on high-risk content, enabling aggressive pruning while preserving task-critical information. Experiments on real-world UI observations derived from WebArena show that MINIM substantially reduces task-irrelevant sensitive leakage while preserving task-critical semantic context and the interactive affordances required for reliable agent actions.

Problem

• privacy leakage
• UI state observation
• sensitive information
• autonomous agents
• task-irrelevant data

Innovation

• privacy-aware minimization
• Contextual Integrity
• UI state sanitization
• LLM-powered agents
• local trusted broker