AgentRewardBench: Evaluating Automatic Evaluations of Web Agent Trajectories

2025-04-11

AI Summary

Web agent evaluation faces two key bottlenecks: poor generalizability of rule-based methods and high cost of human annotation. Method: We introduce AgentRewardBench, the first benchmark for automatic browser-trajectory evaluation, comprising 1,302 expert-annotated trajectories across five task suites and four agent architectures, with fine-grained annotations of success rate, side effects, and redundancy. We systematically evaluate twelve large language models (LLMs) as automated judges—a first-of-its-kind study—and propose a multidimensional evaluation framework with cross-benchmark and cross-model comparison protocols. Contribution/Results: No single LLM dominates all metrics; rule-based baselines underestimate success rates by 17.3% on average. LLM-based judges significantly improve evaluation robustness and scalability. All resources—including benchmark data, annotations, and evaluation protocols—are fully open-sourced to advance reproducible Web agent evaluation research.

Technology Category

• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Multiagent Systems: Teamwork
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Web agents enable users to perform tasks on web browsers through natural language interaction. Evaluating web agents trajectories is an important problem, since it helps us determine whether the agent successfully completed the tasks. Rule-based methods are widely used for this purpose, but they are challenging to extend to new tasks and may not always recognize successful trajectories. We may achieve higher accuracy through human evaluation, but the process would be substantially slower and more expensive. Automatic evaluations with LLMs may avoid the challenges of designing new rules and manually annotating trajectories, enabling faster and cost-effective evaluation. However, it is unclear how effective they are at evaluating web agents. To this end, we propose AgentRewardBench, the first benchmark to assess the effectiveness of LLM judges for evaluating web agents. AgentRewardBench contains 1302 trajectories across 5 benchmarks and 4 LLMs. Each trajectory in AgentRewardBench is reviewed by an expert, who answers questions pertaining to the success, side effects, and repetitiveness of the agent. Using our benchmark, we evaluate 12 LLM judges and find that no single LLM excels across all benchmarks. We also find that the rule-based evaluation used by common benchmarks tends to underreport the success rate of web agents, highlighting a key weakness of rule-based evaluation and the need to develop more flexible automatic evaluations. We release the benchmark at: https://agent-reward-bench.github.io

Problem

• Evaluating web agent trajectories for task success
• Challenges of rule-based and human evaluation methods
• Assessing LLM effectiveness in automatic trajectory evaluation

Innovation

• LLM judges for web agent evaluation
• AgentRewardBench benchmark with expert reviews
• Comparing rule-based and LLM evaluation methods