Two-Stage Prompt Optimization for Few-Shot Relation Extraction: From Reasoning-Guided Search to Gradient-Guided Refinement

2026-06-28

AI Summary

This work addresses the lack of effective prompt optimization methods for small-scale language models in few-shot relation extraction. The authors propose a two-stage prompt optimization framework: the first stage employs reasoning-guided global search to broadly optimize prompts at the natural language level, while the second stage introduces GradPO, a novel gradient-guided algorithm that identifies and refines high-impact prompt segments using loss and gradient signals. This approach uniquely integrates reasoning-based global exploration with gradient-driven local refinement, substantially enhancing prompt quality. Evaluated on FS-TACRED, the method achieves state-of-the-art performance with Qwen3-4B, and demonstrates competitive results on FS-FewRel, confirming the framework’s effectiveness and generalizability.

Abstract

Automatic prompt optimization is still underexplored for episodic few-shot relation extraction with smaller language models. We propose a two-stage framework that combines reasoning-based prompt optimization with gradient-based prompt optimization. The first stage can use any reasoning-based optimizer to make broadprompt improvements in natural language. The second stage applies our GradPO, which uses loss and gradient signals to identify high-impact prompt spans and refine them with local edits. Experiments on FS-TACRED and FS-FewRel show that local refinement usually improves prompts found by the first stage, and GradPO is the most consistent refiner. Our framework achieves state-of-the-art performance on FS-TACRED with Qwen3-4B and remains competitive on FS-FewRel.

Problem

• few-shot relation extraction
• prompt optimization
• small language models
• automatic prompt tuning

Innovation

• two-stage prompt optimization
• few-shot relation extraction
• gradient-guided refinement
• reasoning-guided search
• GradPO