In-Context Black-Box Optimization with Unreliable Feedback

2026-05-07

AI Summary

This work addresses the challenge of effectively leveraging auxiliary feedback—such as expert advice or simulator outputs—in black-box optimization, where such signals are often biased, distorted, or unreliable. The authors propose a feedback-aware contextual black-box optimization method that jointly models historical observations and auxiliary feedback for candidate solutions using a pre-trained Transformer. A structured Bayesian prior is introduced to explicitly capture the accessibility, relevance, and distortion characteristics of diverse feedback sources. At test time, the approach dynamically estimates feedback reliability, enabling robust and efficient cross-task optimization. Experimental results demonstrate consistent superiority over existing baselines on both synthetic and real-world tasks, with the model effectively amplifying useful signals, suppressing misleading information, and offering strong interpretability.

Abstract

Black-box optimization in science and engineering often comes with side information: experts, simulators, pretrained predictors, or heuristics can suggest which candidates look promising. This information can accelerate search, but it can also be biased, input-dependent, or misleading. Feedback-aware BO methods typically handle one task at a time, limiting their ability to generalize over multiple sources of feedback. In-context optimizers address cross-task adaptation, but usually assume that optimization history is the only available signal at test time. We study feedback-informed in-context black-box optimization (FICBO), where a pretrained optimizer conditions on both the observed history and cheap auxiliary feedback for the current candidate set. We introduce a structured feedback prior that models how feedback sources vary in their access, relevance, and distortion relative to the true objective, and use it to pretrain a feedback-aware transformer. At test time, the model estimates source reliability in context by comparing observed objective values with auxiliary signals, improving query selection. On synthetic and real-world tasks, FICBO effectively exploits informative feedback while remaining robust to weak or misleading sources, improving over other baselines. Empirical investigations further illustrate how the model perceives test-time sources, offering insights into its interpretability and decision-making process.

Problem

• black-box optimization
• unreliable feedback
• in-context learning
• auxiliary feedback
• feedback robustness

Innovation

• in-context learning
• black-box optimization
• unreliable feedback
• feedback-aware transformer
• structured feedback prior