SIPDO: Closed-Loop Prompt Optimization via Synthetic Data Feedback

2025-05-26

AI Summary

Existing prompt optimization methods rely on static datasets and lack adaptability to distribution shifts or iterative improvement capabilities. This paper proposes the first closed-loop prompt optimization framework, enabling autonomous, iterative refinement without external annotations or new tasks via synthetic data feedback. The method tightly couples synthetic data generation with gradient-based or search-based prompt optimization, incorporating self-supervised feedback evaluation and a closed-loop control mechanism that allows models to automatically identify prompt weaknesses and continuously enhance performance. Evaluated on question-answering and reasoning benchmarks, the framework significantly outperforms standard prompt tuning approaches. Results demonstrate the effectiveness, generalizability, and practicality of synthetic-data-driven prompt learning, establishing a foundation for adaptive, self-improving prompt optimization in dynamic environments.

Technology Category

• Natural Language Processing: Prompt Engineering / Prompting
• Search and Optimization: Learning to Search
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Prompt quality plays a critical role in the performance of large language models (LLMs), motivating a growing body of work on prompt optimization. Most existing methods optimize prompts over a fixed dataset, assuming static input distributions and offering limited support for iterative improvement. We introduce SIPDO (Self-Improving Prompts through Data-Augmented Optimization), a closed-loop framework for prompt learning that integrates synthetic data generation into the optimization process. SIPDO couples a synthetic data generator with a prompt optimizer, where the generator produces new examples that reveal current prompt weaknesses and the optimizer incrementally refines the prompt in response. This feedback-driven loop enables systematic improvement of prompt performance without assuming access to external supervision or new tasks. Experiments across question answering and reasoning benchmarks show that SIPDO outperforms standard prompt tuning methods, highlighting the value of integrating data synthesis into prompt learning workflows.

Problem

• Optimizing prompts for large language models dynamically
• Integrating synthetic data to improve prompt weaknesses
• Enhancing prompt performance without external supervision

Innovation

• Closed-loop framework for prompt optimization
• Synthetic data reveals prompt weaknesses
• Feedback-driven iterative prompt refinement