optimize_anything: A Universal API for Optimizing any Text Parameter

2026-05-19

AI Summary

This work proposes a general-purpose optimization framework grounded in large language models (LLMs), which unifies diverse optimization problems by iteratively refining textual parameters guided by a scoring function. Eliminating the need for domain-specific algorithms, the approach leverages actionable auxiliary information and a multi-task joint search mechanism to substantially accelerate convergence and enhance final performance, while enabling cross-task transfer and generalization to unseen inputs. Empirical results demonstrate state-of-the-art performance across six heterogeneous tasks: achieving 89.5% accuracy on ARC-AGI, reducing cloud scheduling costs by 40%, generating CUDA kernels that outperform PyTorch in 87% of cases, and surpassing AlphaEvolve on circle packing for n=26. This study establishes, for the first time, that LLM-driven textual optimization can serve as a universal problem-solving paradigm.

Abstract

Can a single LLM-based optimization system match specialized tools across fundamentally different domains? We show that when optimization problems are formulated as improving a text artifact evaluated by a scoring function, a single AI-based optimization system-supporting single-task search, multi-task search with cross-problem transfer, and generalization to unseen inputs-achieves state-of-the-art results across six diverse tasks. Our system discovers agent architectures that nearly triple Gemini Flash's ARC-AGI accuracy (32.5% to 89.5%), finds scheduling algorithms that cut cloud costs by 40%, generates CUDA kernels where 87% match or beat PyTorch, and outperforms AlphaEvolve's reported circle packing solution (n=26). Ablations across three domains reveal that actionable side information yields faster convergence and substantially higher final scores than score-only feedback, and that multi-task search outperforms independent optimization given equivalent per-problem budget through cross-task transfer, with benefits scaling with the number of related tasks. Together, we show for the first time that text optimization with LLM-based search is a general-purpose problem-solving paradigm, unifying tasks traditionally requiring domain-specific algorithms under a single framework. We open-source optimize\_anything with support for multiple backends as part of the GEPA project at https://github.com/gepa-ai/gepa .

Problem

• text optimization
• universal optimization
• LLM-based search
• general-purpose problem-solving
• cross-domain optimization

Innovation

• LLM-based optimization
• text parameter optimization
• cross-task transfer
• universal optimization API
• actionable side information