MemoHarness: Agent Harnesses That Learn from Experience

2026-07-14

AI Summary

This work addresses the limitations of existing agents that rely on fixed global control layers, which hinder adaptability across diverse tasks and constrain performance. The authors propose a learnable, adaptive control framework that constructs the agent’s control layer directly from execution experience, enabling case-level adaptation without requiring labeled data or explicit search. The framework features a six-dimensional editable control structure, an experience storage and retrieval mechanism, and context-aware caching to optimize efficiency. Experimental results demonstrate that this approach outperforms fixed-control baselines on Shell command execution, code generation, and analytical reasoning tasks. Moreover, it exhibits strong transferability to both novel tasks and new underlying models while maintaining computationally tractable inference overhead.

Abstract

An agent harness is the external control layer that turns a base LLM into an executable agent by managing context, tools, orchestration, memory, decoding, and output handling. While harness design strongly affects agent behavior, most automatic improvement methods optimize narrower artifacts such as prompts, pipelines, or workflows, and deployed agents usually reuse a single global harness for all cases. We introduce MemoHarness, an adaptive harness optimization framework that learns from its own executions. MemoHarness decomposes the harness into six editable control dimensions, stores per-case diagnoses and distilled global patterns in a dual-layer experience bank, and adapts the learned harness to each test case using retrieved experience without test-time labels, feedback, or additional search. In our evaluation across shell-agent, code-generation, and analytical-reasoning benchmarks, MemoHarness improves over the fixed harnesses we compare against and shows selective transfer to unseen suites and base models. Its additional context can also remain cost-competitive when much of the retrieved experience is cacheable. These results provide evidence that execution experience is a practical substrate for building agent harnesses that are more adaptive than a single static configuration, while leaving broader claims about statistical robustness and component attribution to future work.

Problem

• agent harness
• adaptive control
• execution experience
• static configuration
• task adaptation

Innovation

• adaptive harness
• experience-based learning
• agent control
• retrieval-augmented execution
• dual-layer memory