From Question Answering to Task Completion: A Survey on Agent System and Harness Design

2026-06-14

AI Summary

This study addresses the question of whether performance bottlenecks of large language model (LLM) agents in long-horizon tasks stem from the base model, the execution framework, or their coupling. To this end, the authors propose a “model–framework” co-analysis perspective, decomposing the execution framework into six runtime responsibilities: observation, context, control, action, state, and verification. They further integrate this decomposition with the four paradigms of agent engineering—prompt engineering, workflow design, context engineering, framework engineering, and co-training—to systematically categorize existing approaches. The resulting analytical framework elucidates how runtime design choices influence task success rate, efficiency, and reliability, while highlighting critical challenges including value-aware evaluation, safety, framework generalization, and co-evolution of models and frameworks.

Abstract

LLM-based agents mark a shift from passive question answering to active task completion: they perceive environments, invoke tools, maintain state, and act over extended horizons. As agent systems have evolved from prompt engineering to workflows and context engineering, harness engineering, and agent-native training with co-evolution, a central question has become increasingly important: where does the bottleneck in agent performance reside, in the foundation model, in the execution harness, or in the coupling between them? This survey examines LLM-based agents through a model-harness lens. We first clarify the functional definition of agents and the implementation view of an LLM-based agent as a foundation model coupled with an execution harness. We then analyze the limits of model-centric scaling, trace four paradigms of agent engineering, and decompose the execution harness into six coupled runtime responsibilities: observation, context, control, action, state, and verification. Using this decomposition, we map task properties and domain pressures to harness configurations, review benchmark and evaluation practices, and synthesize model-harness evidence on how runtime design affects long-horizon task completion, efficiency, and reliability. Finally, we identify open challenges in value-aware evaluation, safety, harness generalization, and model-harness co-evolution. Rather than treating agents as models with auxiliary tools, this survey argues that agent quality -- including success, efficiency, safety, and generalization -- emerges from the interaction between model capability, runtime infrastructure, task structure, and evaluation design. A collection of papers discussed in this survey is provided in https://github.com/ggjy/Awesome-Agent-Engineering.

Problem

• LLM-based agents
• execution harness
• performance bottleneck
• model-harness coupling
• task completion

Innovation

• model-harness coupling
• execution harness decomposition
• LLM-based agents
• runtime responsibilities
• agent engineering paradigms