In-Context Multi-Objective Optimization

2025-12-11

AI Summary

Expensive black-box multi-objective optimization (MOO) faces bottlenecks including task-specific surrogate modeling and acquisition function design, myopic decision-making, and high computational overhead. Method: We propose the first end-to-end, fully amortized general-purpose optimization paradigm: a Transformer-based reinforcement learning policy that directly generates high-quality candidate solutions via forward inference—conditioned solely on the historical query sequence and trained with cumulative hypervolume improvement as reward. Contribution/Results: Our approach eliminates reliance on explicit surrogate fitting and handcrafted acquisition functions, enabling in-context, full Pareto front approximation and multi-step planning for the first time. Evaluated on synthetic and real-world benchmarks, it accelerates proposal generation by 50–1000× over state-of-the-art methods while maintaining or surpassing Pareto solution quality—even under tight evaluation budgets.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Optimization
• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications

Abstract

Balancing competing objectives is omnipresent across disciplines, from drug design to autonomous systems. Multi-objective Bayesian optimization is a promising solution for such expensive, black-box problems: it fits probabilistic surrogates and selects new designs via an acquisition function that balances exploration and exploitation. In practice, it requires tailored choices of surrogate and acquisition that rarely transfer to the next problem, is myopic when multi-step planning is often required, and adds refitting overhead, particularly in parallel or time-sensitive loops. We present TAMO, a fully amortized, universal policy for multi-objective black-box optimization. TAMO uses a transformer architecture that operates across varying input and objective dimensions, enabling pretraining on diverse corpora and transfer to new problems without retraining: at test time, the pretrained model proposes the next design with a single forward pass. We pretrain the policy with reinforcement learning to maximize cumulative hypervolume improvement over full trajectories, conditioning on the entire query history to approximate the Pareto frontier. Across synthetic benchmarks and real tasks, TAMO produces fast proposals, reducing proposal time by 50-1000x versus alternatives while matching or improving Pareto quality under tight evaluation budgets. These results show that transformers can perform multi-objective optimization entirely in-context, eliminating per-task surrogate fitting and acquisition engineering, and open a path to foundation-style, plug-and-play optimizers for scientific discovery workflows.

Problem

• Optimizes multiple competing objectives in black-box systems
• Reduces need for per-task model retraining and tuning
• Accelerates design proposal times for efficient optimization

Innovation

• Transformer architecture for multi-objective optimization
• Pretrained policy for fast design proposals without retraining
• Reinforcement learning to maximize cumulative hypervolume improvement