Who Grades the Grader? Co-Evolving Evaluation Metrics and Skills for Self-Improving LLM Agents

2026-07-14

AI Summary

This work addresses the challenge that existing self-evolving agent systems rely on reliable evaluation metrics, which are often unavailable in real-world settings due to the absence of trustworthy supervisory signals. To overcome this limitation, the paper proposes a self-improvement framework that operates without pre-existing high-quality scoring standards by co-evolving evaluation metrics and a skill library to achieve robust autonomous enhancement. Key innovations include an evolvable, transparent metric mechanism and a dual-ratchet co-evolutionary architecture, integrated with an evolutionary metric loop driven by ensembles of flaw detectors, anchor reference set training, unsupervised output consensus regularization, and an external auditing mechanism. Evaluated on code generation, enterprise-grade text-to-SQL, and reference-free report generation tasks, the approach attains 88–110% of the performance of ground-truth-metric-driven methods and demonstrates verified safety and efficacy under independent human evaluation.

Abstract

Self-evolving agent systems improve by creating, revising, and retiring their own skills, but every such loop rests on a hidden assumption: a reliable evaluation metric already exists. In many real applications it does not. We make three claims. First, metrics can be \emph{evolved}: our metric loop searches compositions of small drawback detectors under a full evolutionary lifecycle, trained to agree with a ten-item anchored reference set, regularized by consensus over unlabeled outputs, and audited against a held-out anchor it never reads, yielding a transparent, inspectable metric rather than an opaque judge. Second, since no metric exists to beat, the yardstick is recovering what an accurate metric would have enabled, and \emph{Double Ratchet}, our co-evolution of the metric with a lifecycle-managed skill loop, does so: across code generation (MBPP+), enterprise text-to-SQL (Spider~2.0-Snow), and reference-free report generation, it retains 88--110\% of the held-out lift achieved by the same skill loop driven by ground truth or the best available rubric. Third, safety comes from anchor discipline plus outer audits: removing anchor guards collapses the metric into a vacuous detector while removing the lifecycle does not; and when evolved skills gamed the report rubric, an independent judge caught it, one detector repaired it, and a task-aware judge then preferred the evolved outputs over the pre-evolution baseline in 77\% of decided pairs. We argue this failure-expecting architecture is the right default wherever no reliable automatic verifier exists.

Problem

• evaluation metrics
• self-improving agents
• LLM agents
• metric evolution
• automatic evaluation

Innovation

• co-evolution
• evaluation metrics
• self-improving agents
• evolutionary lifecycle
• anchor-based auditing