What Makes a Good Terminal-Agent Benchmark Task: A Guideline for Adversarial, Difficult, and Legible Evaluation Design

2026-04-30

AI Summary

Current benchmarks for terminal-based agents predominantly rely on prompt engineering paradigms, lacking adversarial validation logic and thus remaining vulnerable to reward hacking while failing to authentically assess model capabilities. This work presents the first systematic characterization of common failure modes in terminal agent tasks, distinguishing between conceptual difficulty and environmental complexity. It proposes a design principle centered on verifiable objectives, emphasizing that tasks should be adversarial, challenging, and interpretable. Drawing on over a year of task contribution and review experience with Terminal Bench, combined with empirical analysis and categorization of failure cases, the study identifies reward loopholes in more than 15% of tasks in mainstream benchmarks. These findings inform a practical, actionable design guideline for benchmark developers aiming to construct more robust and reliable evaluation suites.

Abstract

Terminal-agent benchmarks have become a primary signal for measuring the coding and system-administration capabilities of large language models. As the market for evaluation environments grows, so does the pressure to ship tasks quickly, often without thorough adversarial review of the verification logic. This paper is a guideline for writing good benchmark tasks, drawn from over a year of contributing to and reviewing tasks for Terminal Bench. Most people write benchmark tasks the way they write prompts. They shouldn't. A prompt is designed to help the agent succeed; a benchmark is designed to find out if it can. We argue that good tasks are adversarial, difficult, and legible, and that a large class of common failure modes -- AI-generated instructions, over-prescriptive specifications, clerical difficulty, oracle solutions that assume hidden knowledge, tests that validate the wrong things, and reward-hackable environments -- are predictable consequences of treating task authoring as prompt authoring. We catalog these failure modes, argue that real difficulty is conceptual rather than environmental, and discuss recent empirical evidence that over 15% of tasks in popular terminal-agent benchmarks are reward-hackable. We hope this serves as a useful reference for benchmark maintainers, task contributors, and researchers using benchmark scores as evidence.

Problem

• terminal-agent benchmark
• adversarial evaluation
• reward hacking
• task design
• evaluation reliability

Innovation

• terminal-agent benchmark
• adversarial evaluation
• reward hacking
• task design guideline
• conceptual difficulty