OSGuard: A Benchmark for Safety in Computer-Use Agents

2026-06-12

AI Summary

This work addresses the critical issue that existing AI agents operating in OS environments often achieve task success through unsafe shortcuts, a risk obscured when evaluating solely by task completion rates. To this end, we propose the first dual-granularity safety evaluation benchmark, which integrates action-level local guardrail assessments with risk-augmented end-to-end execution evaluation to effectively identify behaviors that appear successful but are inherently hazardous. Our framework innovatively combines contextualized action classification, OSWorld-derived task variants, a state-aware safety invariant evaluator, and multimodal guardrail testing techniques. Experimental results demonstrate that while current guardrail models perform adequately on isolated actions, they exhibit significant safety vulnerabilities in complex execution scenarios, thereby underscoring the necessity and diagnostic value of our benchmark.

Abstract

Computer-use agents are increasingly evaluated by whether they complete realistic desktop and web tasks. However, task success alone can miss failures in which an agent reaches the nominal goal through an unsafe shortcut. We introduce OSGuard, a dual-granularity benchmark suite for evaluating safety in computer-use agents under benign, unchanged user instructions. OSGuard contains an action-level benchmark for local guardrail decisions and a risk-augmented execution suite for end-to-end evaluation. The action-level benchmark consists of contextualized proposed actions labeled as allowed, unrelated, or unsafe, each judged relative to the original instruction and current interface state. The execution suite contains manually constructed OSWorld-derived task variants in which the original task remains achievable, but the environment is modified to introduce latent hazards such as destructive overwrites, etc. Each variant is paired with augmented evaluators that retain the original task-success criterion while adding explicit state-based safety invariants, allowing us to distinguish safe completions from unsafe completions that satisfy the nominal task objective. Our experimental results on OSGuard show that current multimodal guardrails can perform well on isolated action judgments, while risk-augmented execution exposes remaining gaps between local oversight and reliable end-to-end safety. This dual-granularity design enables more precise diagnosis of whether models can both recognize unsafe proposed actions and improve full-task safety when deployed as guardrails.

Problem

• computer-use agents
• safety evaluation
• unsafe shortcuts
• benchmark
• task safety

Innovation

• computer-use agents
• safety benchmark
• dual-granularity evaluation
• risk-augmented execution
• guardrail evaluation