ZERA: Zero-init Instruction Evolving Refinement Agent - From Zero Instructions to Structured Prompts via Principle-based Optimization

2025-09-16

AI Summary

Existing automatic prompt optimization (APO) methods optimize only user prompts, relying on unstructured feedback, resulting in high sample and iteration costs and poor robustness. This paper proposes ZERA, the first framework to jointly optimize both system and user prompts, enabling automatic evolution from zero-initialized prompts to structured ones. Its core contributions are threefold: (1) eight generalizable, principle-driven scoring criteria for prompt evaluation; (2) an automated weight inference mechanism coupled with a structured critique generation module; and (3) integration of multi-model comparative analysis and few-shot learning. Extensive experiments across five large language models and nine diverse tasks demonstrate significant improvements over strong baselines. Ablation studies confirm that each component critically enhances prompt quality.

Technology Category

• Natural Language Processing: Prompt Engineering / Prompting
• Search and Optimization: Learning to Search
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Automatic Prompt Optimization (APO) improves large language model (LLM) performance by refining prompts for specific tasks. However, prior APO methods typically focus only on user prompts, rely on unstructured feedback, and require large sample sizes and long iteration cycles-making them costly and brittle. We propose ZERA (Zero-init Instruction Evolving Refinement Agent), a novel framework that jointly optimizes both system and user prompts through principled, low-overhead refinement. ZERA scores prompts using eight generalizable criteria with automatically inferred weights, and revises prompts based on these structured critiques. This enables fast convergence to high-quality prompts using minimal examples and short iteration cycles. We evaluate ZERA across five LLMs and nine diverse datasets spanning reasoning, summarization, and code generation tasks. Experimental results demonstrate consistent improvements over strong baselines. Further ablation studies highlight the contribution of each component to more effective prompt construction. Our implementation including all prompts is publicly available at https://github.com/younatics/zera-agent.

Problem

• Optimizing both system and user prompts for LLMs efficiently
• Reducing costly iterations and large sample requirements in APO
• Improving prompt quality using structured critiques and minimal examples

Innovation

• Jointly optimizes system and user prompts
• Uses eight criteria with inferred weights
• Enables fast convergence with minimal examples