EvoMem: Memory-Augmented Evolution for Code Optimization

2026-08-11

AI Summary

This work addresses the inefficiency of existing large language model (LLM)-based evolutionary code search methods, which discard effective mutation strategies after each run, hindering knowledge reuse across tasks or iterations and leading to redundant exploration. To overcome this limitation, the paper introduces EvoMem—a persistent memory architecture that, for the first time, integrates a structured, task-aware external memory mechanism into LLM-driven evolutionary search. EvoMem captures successful strategies through knowledge extraction, stores them with source annotations, and enables context-aware retrieval, allowing dynamic reuse in subsequent tasks. Experimental results demonstrate that EvoMem significantly improves search efficiency or target performance across diverse domains, including geometric optimization, multi-hop question answering, and GPU kernel optimization, effectively enabling cross-iteration and cross-task transfer of successful strategies.

Abstract

Successful mutation strategies in evolutionary code search may contain reusable knowledge that is useful beyond a single run, and in some cases may transfer across related tasks and domains. However, existing LLM-driven evolutionary frameworks largely discard such knowledge, repeatedly rediscovering similar ideas and limiting opportunities for cross-run and cross-task learning. We introduce EvoMem, a persistent memory architecture for LLM-based evolutionary program search that captures and reuses candidate mutation knowledge. EvoMem converts successful mutation events into structured, task-aware advice for future runs. It operates in two phases: after each run, it extracts and stores promising ideas with provenance, and during subsequent evolution, it retrieves a small set of relevant instructions based on the current task and program context to guide mutation. Across geometric optimization, multi-hop question answering, GPU kernel optimization, and related benchmarks, our experiments show positive average improvements in target metrics or search speed for most evaluated settings, while also revealing variability across tasks. Overall, EvoMem provides evidence that persistent memory can reduce some redundant exploration and improve the reuse and adaptation of successful strategies in LLM-driven evolutionary search.

Problem

• evolutionary code search
• memory reuse
• cross-task learning
• mutation strategies
• LLM-driven optimization

Innovation

• persistent memory
• evolutionary search
• code optimization
• mutation reuse
• LLM-driven evolution