Self-Play Only Evolves When Self-Synthetic Pipeline Ensures Learnable Information Gain

2026-02-10

AI Summary

Existing self-evolution systems for large language models often plateau rapidly due to synthetic data lacking sufficient learnable information gain. This work proposes a triadic role-based self-evolution framework comprising a proposer, a solver, and a verifier, which sustains high information gain across iterative cycles through asymmetric weak–strong–weak co-evolution, dynamic model capacity expansion, and active incorporation of external knowledge. By centering the system design on learnable information gain, this study achieves, for the first time, sustainable self-evolution at the architectural level. Empirical validation on self-play programming tasks demonstrates the framework’s capability to consistently surpass performance plateaus and enable stable, continuous capability improvement.

Abstract

Large language models (LLMs) make it plausible to build systems that improve through self-evolving loops, but many existing proposals are better understood as self-play and often plateau quickly. A central failure mode is that the loop synthesises more data without increasing learnable information for the next iteration. Through experiments on a self-play coding task, we reveal that sustainable self-evolution requires a self-synthesised data pipeline with learnable information that increases across iterations. We identify triadic roles that self-evolving LLMs play: the Proposer, which generates tasks; the Solver, which attempts solutions; and the Verifier, which provides training signals, and we identify three system designs that jointly target learnable information gain from this triadic roles perspective. Asymmetric co-evolution closes a weak-to-strong-to-weak loop across roles. Capacity growth expands parameter and inference-time budgets to match rising learnable information. Proactive information seeking introduces external context and new task sources that prevent saturation. Together, these modules provide a measurable, system-level path from brittle self-play dynamics to sustained self-evolution.

Problem

• self-evolution
• learnable information gain
• self-play
• large language models
• data synthesis

Innovation

• self-evolution
• learnable information gain
• asymmetric co-evolution
• capacity growth
• proactive information seeking