Towards Policy-Compliant Agents: Learning Efficient Guardrails For Policy Violation Detection

2025-10-03

AI Summary

Autonomous web agents frequently violate human-specified policies during long-horizon tasks, yet existing work lacks cross-domain, fine-grained evaluation of policy compliance. Method: We introduce PolicyGuardBench—the first large-scale, multi-domain benchmark for policy compliance assessment—supporting both cross-subdomain and cross-domain violation detection. It pioneers a prefix-based violation prediction task to enable early warning on incomplete agent trajectories. Leveraging diverse agent execution traces, we construct a meticulously annotated violation dataset and design PolicyGuard-4B, a lightweight detection model. Results: Experiments demonstrate that PolicyGuard-4B achieves high accuracy in both full-trajectory violation detection and prefix-based prediction, exhibits strong generalization, and maintains superior performance on unseen domains and subdomains. The framework significantly enhances long-horizon policy adherence, establishing a new standard for evaluating and improving policy-aware web agents.

Technology Category

• Humans and AI: Human-Aware Planning and Behavior Prediction
• Multiagent Systems: Adversarial Agents
• Application Domains: Web

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Autonomous web agents need to operate under externally imposed or human-specified policies while generating long-horizon trajectories. However, little work has examined whether these trajectories comply with such policies, or whether policy violations persist across different contexts such as domains (e.g., shopping or coding websites) and subdomains (e.g., product search and order management in shopping). To address this gap, we introduce PolicyGuardBench, a benchmark of about 60k examples for detecting policy violations in agent trajectories. From diverse agent runs, we generate a broad set of policies and create both within subdomain and cross subdomain pairings with violation labels. In addition to full-trajectory evaluation, PolicyGuardBench also includes a prefix-based violation detection task where models must anticipate policy violations from truncated trajectory prefixes rather than complete sequences. Using this dataset, we train PolicyGuard-4B, a lightweight guardrail model that delivers strong detection accuracy across all tasks while keeping inference efficient. Notably, PolicyGuard-4B generalizes across domains and preserves high accuracy on unseen settings. Together, PolicyGuardBench and PolicyGuard-4B provide the first comprehensive framework for studying policy compliance in web agent trajectories, and show that accurate and generalizable guardrails are feasible at small scales.

Problem

• Detecting policy violations in autonomous web agent trajectories
• Evaluating compliance across different domains and subdomains
• Developing lightweight guardrail models for efficient violation detection

Innovation

• PolicyGuardBench benchmark detects agent policy violations
• PolicyGuard-4B model provides efficient violation detection
• Lightweight guardrail generalizes across domains and settings