EvoPolicyGym: Evaluating Autonomous Policy Evolution in Interactive Environments

2026-07-02

AI Summary

Existing evaluation methods struggle to assess autonomous agents’ ability to continuously refine executable policies through feedback under a limited interaction budget. To address this gap, this work proposes a novel paradigm termed “autonomous policy evolution” and introduces EvoPolicyGym—a controlled, interactive reinforcement learning benchmark that enables agents to iteratively edit and optimize their policies within a fixed budget while offering trajectory-level diagnostic capabilities to analyze budget allocation and feedback utilization mechanisms. Experimental results demonstrate that agents augmented with large language models (e.g., GPT-5.5) achieve top-two performance across all 16 environments and obtain the highest aggregate ranking, validating the framework’s effectiveness in evaluating policy evolution capabilities.

Abstract

Autonomous agents are increasingly expected to improve executable policies through feedback, yet existing evaluations often collapse this process into a final score or confound it with open-ended software-engineering progress. We introduce Autonomous Policy Evolution, a controlled evaluation setting in which a harness-model agent repeatedly edits an executable policy system under a fixed interaction budget. We instantiate this setting in EvoPolicyGym, a benchmark built from compact interactive RL environments that evaluates how agents iteratively improve explored policies. On the EvoPolicyGym suite, GPT-5.5 achieves the strongest aggregate rank score and top-two performance on all 16 environments. Beyond leaderboard results, EvoPolicyGym also provides trajectory-level diagnostics that distinguish how agents allocate budget, convert feedback into parametric tuning. These analyses show that strong autonomous policy evolution depends not only on isolated task wins, but on discovering task-appropriate mechanisms and refining policies under bounded feedback.

Problem

• Autonomous Policy Evolution
• interactive environments
• policy improvement
• evaluation benchmark
• feedback utilization

Innovation

• Autonomous Policy Evolution
• EvoPolicyGym
• interactive reinforcement learning
• policy iteration
• feedback-constrained optimization