$p1$: Better Prompt Optimization with Fewer Prompts

2026-04-09

AI Summary

Existing prompt optimization methods struggle to effectively evaluate the quality of system prompts under heterogeneous user prompts and exhibit substantial performance variance. This work identifies, through variance decomposition, that the magnitude of response variance across system prompts is critical to successful optimization. Building on this insight, the authors propose the p1 method, which selectively leverages only a small set of high-variance user prompts for optimization—enabling efficient discrimination of system prompt quality without updating model weights. Evaluated on reasoning benchmarks, p1 significantly outperforms strong baselines such as full-data training and GEPA, achieving exceptional generalization with just two prompts from AIME 2024, thereby substantially enhancing both optimization efficiency and scalability.

Technology Category

• Search and Optimization: Learning to Search
• Natural Language Processing: Prompt Engineering / Prompting
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Prompt optimization improves language models without updating their weights by searching for a better system prompt, but its effectiveness varies widely across tasks. We study what makes a task amenable to prompt optimization. We show that the reward variance across different system prompts can be decomposed into two components: variance among responses, which captures generation stochasticity, and variance among system prompts, which captures differences in system prompt quality. Prompt optimization succeeds when variance among system prompts is sufficiently large, but fails when variance among responses dominates the variance of the system prompts. Surprisingly, we further show that scaling to more user prompts can hurt optimization by reducing variance among system prompts, especially on heterogeneous datasets where different user prompts favor different system prompts. Motivated by this insight, we propose $p1$, a simple user prompt filtering method that selects a small subset of user prompts with high variance across candidate system prompts. This subset of user prompts allows one to distinguish a good system prompt from a bad one, making system optimization easier. Experiments on reasoning benchmarks show that $p1$ substantially improves prompt optimization over training on the full dataset and outperforms strong baselines such as GEPA. Notably, training on only two prompts from AIME 24 yields a system prompt that generalizes well to other reasoning benchmarks.

Problem

• prompt optimization
• system prompt
• reward variance
• user prompts
• heterogeneous datasets

Innovation

• prompt optimization
• system prompt
• reward variance
• user prompt filtering
• few-shot prompting