OpenLoopEvolve: A Verifiable Self-Evolution Framework for Loop Policies in Long-Horizon Complex Tasks

2026-08-10

AI Summary

In long-horizon, complex tasks, agents struggle to accumulate and reuse control experience across tasks, limiting policy generalization and continuous improvement. To address this, this work proposes OpenLoopEvolve, a framework that, for the first time, encapsulates an agent’s observation, planning, memory, and action capabilities into versioned and governable Loop Policy assets. The framework incorporates lineage tracking, large-model-driven autonomous policy generation, a Champion-Challenger evaluation mechanism, and runtime rollback strategies to enable dual-mode (online and offline) self-evolution. Experiments on the YC-Bench benchmark demonstrate significant improvements in task success rate, overall performance, and risk mitigation, validating the efficacy of Loop Policy as an evolvable and reusable policy asset.

Abstract

Long-horizon complex tasks require agents to repeatedly observe states, formulate plans, invoke tools, verify results, and recover from failures in continuously changing environments. However, such control experience often remains confined to a single context or a fixed prompt, and is difficult to accumulate and reuse across historical traces. This paper presents OpenLoopEvolve (OLE), a self-evolution framework centered on the Loop Policy. OLE represents an agent's observation, planning, memory, action, verification, recovery, stopping, and budget control as portable policy assets with versions and lineages, and provides online and offline evolution modes that can be selected according to practical needs: the online mode triggers candidate generation based on feedback from continuous operation, whereas the offline mode searches for candidate policies from archived traces and failure evidence. Both modes share an evolution mechanism consisting of autonomous proposals by a large language model, Champion--Challenger paired evaluation, and robust release. Policies released online are activated at a subsequent task boundary, monitored using subsequent feedback, and rolled back to their parent versions when degradation conditions are met. On the simulated business benchmark YC-Bench, both modes improve aggregate task performance, task success rate, and risk metrics relative to a fixed initial Loop Policy. The results indicate that treating the Loop Policy as a governable asset can support the accumulation, comparison, release, and reuse of control experience and improve agent performance on long-horizon complex tasks.

Problem

• long-horizon tasks
• loop policy
• experience accumulation
• policy reuse
• complex task automation

Innovation

• Loop Policy
• Self-Evolution
• Verifiable Framework
• Champion-Challenger Evaluation
• Policy Versioning