Problem
Research questions and friction points this paper is trying to address.
personalized agents
computer-use benchmark
web tasks
digital life
logged-in environments
MyPCBench: A Benchmark for Personally Intelligent Computer-Use Agents
🤖 AI Summary
Existing agent benchmarks inadequately model personalized scenarios—such as user context, historical data, and authentication—limiting their ability to evaluate real-world digital assistant capabilities. To address this gap, this work proposes MyPCBench, the first benchmark framework for personalized computer-use agents in comprehensive digital life settings. Built on a Linux desktop environment, MyPCBench integrates 17 simulated web applications and a full desktop stack, featuring 184 tasks derived from authentic user requests centered around a unified persona (Michael Scott). The benchmark supports evaluation of cross-application workflows, long-horizon trajectories, and tasks requiring login credentials. Experimental results show that Claude Opus 4.6 achieves the best performance, fully solving 55.4% of tasks, while most models still struggle significantly with complex personalized scenarios. The environment, task suite, and unified tool interface are publicly released.
📝 Abstract
Current benchmarks for computer-use agents evaluate models in impersonal environments. This leaves a gap between evaluation and deployment where personal assistants are expected to work across a user's whole digital life, including their context, historical data, and logged-in accounts. This gap is widest on web tasks, where live web evaluations cannot exercise sites that require logging in or personal information, the kind of site a real personal assistant has to drive. We introduce MyPCBench, which tests computer-use agents as personal assistants on a Linux desktop populated with 17 simulated real-world web applications and a full desktop stack, all seeded for one canonical persona, Michael Scott from The Office. We define 184 tasks in this environment, each inspired by a real request drawn from the OpenClaw community, and benchmark six closed and open-weight models with a uniform computer+bash tool surface. We find that the best model, Claude Opus 4.6, fully solves 55.4\% of the tasks, the only model above 50\%. Model failures cluster on tasks that span many applications and on long trajectories, where personalization stresses an assistant the most. We release the environment, task set, and agent harness at https://mypcbench.com.
Innovation
Methods, ideas, or system contributions that make the work stand out.
personalized agents
computer-use benchmark
web automation
multi-application tasks
agent evaluation
🔎 Similar Papers
No similar papers found.
