From Procedural Skills to Strategy Genes: Towards Experience-Driven Test-Time Evolution

2026-04-16

AI Summary

This work addresses the challenge of effectively representing reusable experience to support stable control and iterative evolution during testing. It proposes a “Policy Gene” representation that encodes experience in a compact, controllable, and evolvable structure, outperforming conventional document-oriented skill bundles. The approach integrates experience representation learning, structured encoding, test-time evolution mechanisms, and failure-aware knowledge distillation. Evaluated across 4,590 experiments spanning 45 scientific code-solving scenarios, the method demonstrates significant gains: on the CritPt benchmark, the gene-based evolution system improves base model performance from 9.1% to 18.57% and from 17.7% to 27.14%, substantially enhancing model adaptability and robustness.

Technology Category

• Machine Learning: Evolutionary Learning
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies

Abstract

This beta technical report asks how reusable experience should be represented so that it can function as effective test-time control and as a substrate for iterative evolution. We study this question in 4.590 controlled trials across 45 scientific code-solving scenarios. We find that documentation-oriented Skill packages provide unstable control: their useful signal is sparse, and expanding a compact experience object into a fuller documentation package often fails to help and can degrade the overall average. We further show that representation itself is a first-order factor. A compact Gene representation yields the strongest overall average, remains competitive under substantial structural perturbations, and outperforms matched-budget Skill fragments, while reattaching documentation-oriented material usually weakens rather than improves it. Beyond one-shot control, we show that Gene is also a better carrier for iterative experience accumulation: attached failure history is more effective in Gene than in Skill or freeform text, editable structure matters beyond content alone, and failure information is most useful when distilled into compact warnings rather than naively appended. On CritPt, gene-evolved systems improve over their paired base models from 9.1% to 18.57% and from 17.7% to 27.14%. These results suggest that the core problem in experience reuse is not how to supply more experience, but how to encode experience as a compact, control-oriented, evolution-ready object.

Problem

• experience reuse
• test-time control
• iterative evolution
• compact representation
• evolution-ready object

Innovation

• Gene representation
• test-time evolution
• experience reuse
• compact encoding
• iterative learning