RLMOpt: Adaptive Prompt Optimization via Recursive Language Models

2026-08-11

AI Summary

Existing prompt optimization methods rely on predefined search strategies, limiting their adaptability to diverse task requirements. This work proposes RLMOpt, the first approach that delegates prompt search strategy to a recursive language model (RLM) for dynamic decision-making. In a tool-augmented environment, the RLM acts as an agent that autonomously analyzes tasks, generates candidate prompts, allocates evaluation budgets, and determines termination conditions. The method integrates deterministic evaluation, Pareto selection, and regression constraints to ensure efficient and reliable optimization. Evaluated across four benchmark tasks, RLMOpt outperforms baselines in 9 out of 11 comparisons, consistently matches or exceeds the performance of initial prompts, achieves comparable or superior results with fewer search iterations, and produces prompts whose lengths range from 27% to 79% of those generated by the GEPA baseline.

Abstract

Prompt optimizers automate the search for prompts that improve language-model performance, but existing methods rely on a predefined optimization procedure: the algorithm determines which candidates to explore and how the search progresses, while the language model generates or refines prompt proposals. We introduce RLMOpt, a prompt optimizer that makes the search policy itself language-model-driven through a recursive language model (RLM). The RLM agent operates over a tool-based environment, inspecting task information, analyzing failures, generating candidates, allocating evaluation budget, and deciding when to stop. A deterministic harness complements the agent by enforcing objective scoring, Pareto-based selection, and regression constraints. We evaluate RLMOpt across four benchmarks spanning structured clinical information extraction (Chia), multi-hop question answering (HotpotQA), verifiable instruction following (IFBench-2025), and multi-turn tool-calling agents (BFCL). In a matched comparison at a single seed, RLMOpt obtains the best held-out score on all four benchmarks and leads the four-task mean (0.610 against 0.589 for GEPA). Repeating each benchmark across seeds yields 11 matched benchmark-seed comparisons, in which RLMOpt outperforms GEPA in 9 cases. Across all 11 runs, it never produced a prompt that underperformed its seed, whereas GEPA fell below its starting point twice. It is also more efficient, achieving these results with fewer search rollouts while producing prompts that are 27-79% the size of those produced by GEPA. Our results further show that optimization gains are determined primarily by the headroom available in the seed prompt, rather than by the search budget. Efficient optimization therefore depends on reaching the available headroom reliably and with minimal search

Problem

• prompt optimization
• language models
• search policy
• adaptive prompting
• recursive language models

Innovation

• Recursive Language Model
• Prompt Optimization
• Adaptive Search Policy
• Tool-based Environment
• Pareto-based Selection