LemonHarness Technical Report

2026-06-23

AI Summary

This work addresses the instability in long-horizon large language model (LLM) agents caused by dispersed and untraceable state changes due to the absence of well-defined workspace boundaries. To mitigate this, the authors propose an integrated execution framework that explicitly delineates workspace boundaries and unifies model invocations, tool executions, and rule-based knowledge through a structured interface for managing all state transitions. The framework incorporates a reusable rule knowledge base and a time-aware scheduling mechanism, enabling agents to dynamically adapt their strategies. Evaluated on Terminal-Bench 2.0, the LemonHarness_GPT-5.3-CodeX agent achieves an accuracy of 84.49%; when augmented with GPT-5.5, it attains an average accuracy of 86.52% across five tasks, demonstrating significantly improved controllability and stability in long-horizon settings.

Abstract

As large language model (LLM) agents are applied to longer tasks, they increasingly modify workspace state across multiple rounds of iteration. However, agents typically observe only tool outputs and log fragments, while the actual state changes occur in the file system. Without explicit workspace boundaries, state-changing operations such as file writes and temporary artifact generation may scatter changes across paths. Over time, these weakly constrained changes accumulate, making states such as modified files difficult to track. This paper presents LemonHarness, an integrated execution framework for long-horizon agents. LemonHarness establishes an explicit execution boundary by constraining state-changing operations within a clearly defined workspace and bringing model invocation, tool execution, and rule knowledge within a single controlled boundary. State-changing operations, including file writes, dependency installation, and temporary artifact creation, are executed through structured tool interfaces, with execution feedback recorded as observations available to subsequent model decisions. The system also introduces a reusable rule knowledge base, which turns recurring execution rules and acceptance criteria into runtime knowledge. LemonHarness further adds a time-aware execution mechanism that exposes elapsed and remaining budget to the model, so it can rebalance exploration, implementation, and validation effort as time pressure shifts and avoid timeouts from long waits or excessive verification. On Terminal-Bench 2.0, LemonHarness_GPT-5.3-CodeX reached 84.49% accuracy over 445 trials; pairing the same framework with the stronger GPT-5.5 backbone raised the average accuracy to 86.52% across five jobs. The results suggest that a unified runtime boundary, callable rule knowledge, and time-aware execution can improve the stability of long-horizon agent execution.

Problem

• long-horizon agents
• workspace state
• state-changing operations
• execution boundary
• temporal constraints

Innovation

• workspace boundary
• rule knowledge base
• time-aware execution
• structured tool interface
• long-horizon agent