Towards the Harness of Embodied Agents

2026-08-03

AI Summary

This work addresses the challenge of effectively extending the successful “harness” paradigm from code-based agents to embodied agents in the physical world, where environmental states are not directly observable and action outcomes are difficult to automatically evaluate. To overcome these limitations, the authors propose Thea, a framework that encapsulates robotic capabilities as modular tools, constructs a symbolic scene graph to serve as contextual representation, and introduces an exit-code–based mechanism for action evaluation and diagnosis, thereby establishing a closed-loop control architecture for embodied intelligence. Thea represents the first successful adaptation of the harness paradigm to real-world physical environments, demonstrating autonomous execution of complex, long-horizon tasks and validating the approach’s feasibility and effectiveness. Furthermore, the compositional use of tools enables emergent, rich behavioral repertoires.

Abstract

The success of coding agents has established the harness as a paradigm: what an agent achieves depends not on the model alone, but on the infrastructure around it. We ask whether the same paradigm extends to embodied agents in the physical world. We present Thea, a harness in which an agentic loop orchestrates robot capabilities, each wrapped as a callable tool. It inherits the core components of coding agents, modified as the physical world requires. The world, however, withholds two abilities that software grants for free: reading the state of the world, and judging the outcome of an action. To bridge these gaps, Thea introduces Scene Graph as Context, a persistent, symbolic representation of the world, and Evaluation as Exit Codes, which detects when an action should terminate, judges whether it succeeded, and on failure diagnoses the cause. Together they close the loop between the agent and the physical world. Rich behaviors then emerge from the composition of tools, and the closed loop carries long-horizon tasks to completion in real environments.

Problem

• embodied agents
• harness
• scene graph
• action evaluation
• physical world

Innovation

• embodied agents
• scene graph
• evaluation as exit codes
• agentic loop
• tool composition