$\varepsilon$-MemEvo: Adaptive Cross-Task Memory Transfer for LLM Program Evolution

2026-08-12

AI Summary

This work addresses the inefficiency of existing large language model–driven program evolution systems, which lack mechanisms for cross-task knowledge reuse, leading to redundant search efforts. To overcome this limitation, the authors propose a cross-task memory transfer framework that distills successful strategies into task-agnostic natural language tactical memories. An adaptive gating mechanism is introduced to dynamically determine when and how to inject these memories during optimization. This approach achieves, for the first time, task-independent representation and safe transfer of tactical knowledge. Evaluated across eight optimization benchmarks, the method yields an average 8.7% improvement in Area Under the Convergence Curve (AUCC), accelerates early-stage convergence by 9.4%, and incurs less than 1% computational overhead. Ablation studies further confirm its robustness and resistance to negative transfer.

Abstract

LLM-based program evolution systems such as FunSearch and AlphaEvolve have shown strong ability to discover novel algorithms, but typically optimize each task in isolation, discarding search experience after completion. We introduce $\varepsilon$-MemEvo, a framework for cross-task knowledge transfer in LLM program evolution. $\varepsilon$-MemEvo stores prior experience as task-agnostic tactic memories: compact natural-language summaries of successful algorithmic strategies rather than raw code, enabling transfer across tasks with different APIs and evaluators. To avoid negative transfer from semantically mismatched memories, $\varepsilon$-MemEvo uses an adaptive injection gate that decides whether retrieved memories should be injected, and at what intensity. We evaluate $\varepsilon$-MemEvo on 8 diverse optimization benchmarks spanning mathematical optimization and systems engineering, using a content-level Leave-One-Out protocol that excludes target-task memory entries. On the primary GPT-5 backbone, $\varepsilon$-MemEvo improves AUCC over AdaEvolve on all 8 tasks, with a mean relative gain of +8.7%, and improves early-stage convergence by +9.4% on average. Ablations show that naive memory injection can fail catastrophically, while adaptive gating remains safe across all five ablation tasks. The data-updated posterior is interpretable in observed states: it favors skip during improving search and shifts from skip to hint across early and late plateaus. These gains incur less than 1% computational overhead.

Problem

• LLM program evolution
• cross-task knowledge transfer
• memory transfer
• algorithmic strategy reuse
• negative transfer

Innovation

• cross-task transfer
• tactic memory
• adaptive gating
• LLM program evolution
• memory injection