The Red Queen Gödel Machine: Co-Evolving Agents and Their Evaluators

2026-06-24

AI Summary

Existing self-improving agents are constrained by static evaluation criteria, hindering their adaptation in dynamic environments where both the agent and evaluation standards co-evolve. This work proposes the Red Queen Gödel Machine (RQGM), which for the first time integrates a non-stationary utility function with recursive self-improvement. By periodically fixing and updating the utility function in stages, RQGM enables co-evolution between the agent and its evaluator. The approach incorporates adversarial objective optimization, an agent-as-a-judge paradigm, and a controllable utility evolution mechanism. Experiments demonstrate that RQGM achieves higher test pass rates in code generation using fewer tokens, significantly improves acceptance rates and scoring accuracy in academic paper review and olympiad proof tasks, and effectively mitigates the over-acceptance bias commonly observed in AI-assisted paper evaluation.

Abstract

Self-improving agents are state-of-the-art (SOTA) on agentic coding benchmarks and have recently been extended to general domains. However, their search methods generally assume a stationary evaluation criterion: a fixed verifier, benchmark, or labeled dataset that remains valid as the agent improves. This ignores a central feature of evolution: species adapt as their environments change with them. We aim to bring the same principle to recursive self-improvement, making evaluation part of the improvement loop and opening search to evolving evaluators, adversarial objectives, and dynamic utilities that may surpass static benchmarks. We introduce the Red Queen Godel Machine (RQGM), an evolutionary framework for recursive self-improvement under non-stationary utilities. The RQGM makes this possible through controlled utility evolution: search is organized into epochs with a fixed within-epoch evaluation criterion, while the utility can be updated at epoch boundaries, so self-improvement guarantees hold per epoch as the objective evolves across them. We begin by showing that even on verifiable coding tasks, the RQGM improves test pass rate over the prior SOTA by adding a complementary agent-as-a-judge code-review signal. This signal is cheaper and the RQGM uses 1.35x-1.72x fewer tokens. We then turn to scientific paper writing and reviewing, and Olympiad-level proof writing and grading, where the RQGM improves performance over prior self-improving agents: co-evolved writers reach 1.78x-1.86x higher acceptance rates under a diverse agent-as-a-judge panel, while co-evolved graders reach 9% higher ground-truth accuracy. In paper reviewing, the strongest baseline reviewer over-accepts AI-generated papers at up to 1.91x the human rate. The RQGM corrects this by introducing an adversarial objective that discovers reviewers equally stringent on AI and human work.

Problem

• recursive self-improvement
• non-stationary evaluation
• co-evolving agents
• dynamic utilities
• Red Queen dynamics

Innovation

• Red Queen Gödel Machine
• co-evolving evaluators
• non-stationary utility
• recursive self-improvement
• adversarial evaluation