On Unsupervised Prompt Learning for Classification with Black-box Language Models

2024-10-04

arXiv.org

AI Summary

To address the high cost of manual annotation or parameter fine-tuning in black-box large language model (LLM) classification tasks, this paper proposes an unsupervised prompt learning method that jointly optimizes discrete prompt templates and pseudo-labels derived from unlabeled data. Its core innovation lies in the first integration of in-context learning (ICL) capability into prompt training—ensuring alignment between prompt construction and inference phases. Specifically, the LLM self-generates high-confidence pseudo-labels to serve as ICL demonstrations, enabling self-supervised prompt optimization. The method operates exclusively via black-box API calls, requiring neither gradient access nor human annotations. Extensive evaluation across multiple benchmark datasets demonstrates substantial improvements in classification accuracy and yields high-quality pseudo-label sets directly usable for downstream supervised fine-tuning.

Technology Category

• Natural Language Processing: Prompt Engineering / Prompting
• Machine Learning: Unsupervised & Self-Supervised Learning
• Search and Optimization: Learning to Search

Application Category

• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Large language models (LLMs) have achieved impressive success in text-formatted learning problems, and most popular LLMs have been deployed in a black-box fashion. Meanwhile, fine-tuning is usually necessary for a specific downstream task to obtain better performance, and this functionality is provided by the owners of the black-box LLMs. To fine-tune a black-box LLM, labeled data are always required to adjust the model parameters. However, in many real-world applications, LLMs can label textual datasets with even better quality than skilled human annotators, motivating us to explore the possibility of fine-tuning black-box LLMs with unlabeled data. In this paper, we propose unsupervised prompt learning for classification with black-box LLMs, where the learning parameters are the prompt itself and the pseudo labels of unlabeled data. Specifically, the prompt is modeled as a sequence of discrete tokens, and every token has its own to-be-learned categorical distribution. On the other hand, for learning the pseudo labels, we are the first to consider the in-context learning (ICL) capabilities of LLMs: we first identify reliable pseudo-labeled data using the LLM, and then assign pseudo labels to other unlabeled data based on the prompt, allowing the pseudo-labeled data to serve as in-context demonstrations alongside the prompt. Those in-context demonstrations matter: previously, they are involved when the prompt is used for prediction while they are not involved when the prompt is trained; thus, taking them into account during training makes the prompt-learning and prompt-using stages more consistent. Experiments on benchmark datasets show the effectiveness of our proposed algorithm. After unsupervised prompt learning, we can use the pseudo-labeled dataset for further fine-tuning by the owners of the black-box LLMs.

Problem

• Optimizing prompts for refining pseudo-supervision without human oversight
• Mitigating overfitting in LLMs using in-context learning techniques
• Enhancing model performance via gradient-based prompt and response refinement

Innovation

• Optimize prompts using pseudo-supervised demonstrations
• Refine pseudo-supervision via gradient-based textual critiques
• Mitigate overfitting with geometric clustering structure