Self-Supervised Prompt Optimization

2025-02-07

AI Summary

Existing prompt optimization methods heavily rely on external reference data—such as ground-truth answers or human annotations—hindering practical deployment. Method: We propose the first fully self-supervised prompt optimization framework that requires no ground truth or human feedback: it leverages large language models (LLMs) themselves to perform pairwise preference comparisons over model outputs, generating implicit optimization signals to guide iterative prompt search and refinement. Contribution/Results: Our approach unifies open-domain and closed-domain tasks, achieving zero-reference, few-shot (only three examples), and low-compute prompt optimization—reducing computational cost to just 1.1%–5.6% of state-of-the-art methods. Evaluated across multiple reasoning benchmarks, it matches or surpasses current best methods, significantly advancing the automation and practicality of LLM prompting.

Technology Category

• Search and Optimization: Learning to Search
• Natural Language Processing: Prompt Engineering / Prompting
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Well-designed prompts are crucial for enhancing Large language models' (LLMs) reasoning capabilities while aligning their outputs with task requirements across diverse domains. However, manually designed prompts require expertise and iterative experimentation. While existing prompt optimization methods aim to automate this process, they rely heavily on external references such as ground truth or by humans, limiting their applicability in real-world scenarios where such data is unavailable or costly to obtain. To address this, we propose Self-Supervised Prompt Optimization (SPO), a cost-efficient framework that discovers effective prompts for both closed and open-ended tasks without requiring external reference. Motivated by the observations that prompt quality manifests directly in LLM outputs and LLMs can effectively assess adherence to task requirements, we derive evaluation and optimization signals purely from output comparisons. Specifically, SPO selects superior prompts through pairwise output comparisons evaluated by an LLM evaluator, followed by an LLM optimizer that aligns outputs with task requirements. Extensive experiments demonstrate that SPO outperforms state-of-the-art prompt optimization methods, achieving comparable or superior results with significantly lower costs (e.g., 1.1% to 5.6% of existing methods) and fewer samples (e.g., three samples). The code is available at https://github.com/geekan/MetaGPT.

Problem

• Automates prompt optimization for LLMs
• Eliminates need for external reference data
• Reduces costs and sample requirements

Innovation

• Self-Supervised Prompt Optimization
• LLM evaluator pairwise comparison
• Cost-efficient prompt discovery framework