Advanced Black-Box Tuning of Large Language Models with Limited API Calls

2025-11-13

AI Summary

Existing black-box tuning methods for large language models (LLMs) without parameter access suffer from either low efficiency, limited performance gains, or prohibitively high API query costs. Method: This paper proposes an efficient, ultra-low-query tuning framework comprising (1) a small-shot LogitMap Pairs querying strategy to extract highly informative output pairs, and (2) a high-fidelity surrogate model based on Gaussian processes (GPs) that accurately approximates the LLM’s logit distribution. Contribution/Results: With only 1.38% of the total API queries required by baseline methods, our approach boosts task accuracy from 55.92% to 86.85%. It outperforms offline, API-free methods and matches the performance of high-query-cost tuning approaches—achieving, for the first time, a principled balance between high accuracy and ultra-low query overhead under stringent query budgets.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Learning to Search
• Natural Language Processing: (Large) Language Models

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Black-box tuning is an emerging paradigm for adapting large language models (LLMs) to better achieve desired behaviors, particularly when direct access to model parameters is unavailable. Current strategies, however, often present a dilemma of suboptimal extremes: either separately train a small proxy model and then use it to shift the predictions of the foundation model, offering notable efficiency but often yielding limited improvement; or making API calls in each tuning iteration to the foundation model, which entails prohibitive computational costs. Therefore, we propose a novel advanced black-box tuning method for LLMs with limited API calls. Our core strategy involves training a Gaussian Process (GP) surrogate model with"LogitMap Pairs"derived from querying the foundation model on a minimal but highly informative training subset. This surrogate can approximate the outputs of the foundation model to guide the training of the proxy model, thereby effectively reducing the need for direct queries to the foundation model. Extensive experiments verify that our approach elevates pre-trained language model accuracy from 55.92% to 86.85%, reducing the frequency of API queries to merely 1.38%. This significantly outperforms offline approaches that operate entirely without API access. Notably, our method also achieves comparable or superior accuracy to query-intensive approaches, while significantly reducing API costs. This offers a robust and high-efficiency paradigm for language model adaptation.

Problem

• Optimizing black-box tuning of LLMs with minimal API query costs
• Balancing efficiency and accuracy in foundation model adaptation
• Reducing computational expense while maintaining performance improvements

Innovation

• Trains Gaussian Process surrogate with LogitMap Pairs
• Reduces API calls by guiding proxy model training
• Achieves high accuracy with minimal foundation model queries