Bayesian Anytime Pareto Set Identification for Multi-Objective Multi-Armed Bandits

2026-06-17

AI Summary

This work addresses the challenge of efficiently identifying the Pareto optimal solution set in multi-objective decision-making by proposing TTPFTS, the first anytime Bayesian multi-objective multi-armed bandit algorithm. Integrating Thompson sampling with a novel uncertainty quantification metric, TTPFTS dynamically assesses the confidence of the estimated Pareto front and is theoretically guaranteed to be asymptotically correct. Empirical evaluations demonstrate that TTPFTS significantly outperforms existing fixed-budget algorithms in both synthetic benchmarks and large-scale molecular discovery tasks. The proposed confidence metric effectively reflects true algorithmic performance, underscoring the method’s practical utility and robustness across diverse problem settings.

Abstract

Identifying Pareto optimal solutions is critical to support multi-objective decision-making. We introduce the first anytime Multi-Objective Multi-Armed Bandit algorithm for the Pareto Set Identification problem, taking a Bayesian approach: Top-Two Pareto Front Thompson Sampling (TTPFTS). We benchmark TTPFTS against state-of-the-art fixed-budget Pareto Set Identification algorithms on synthetic environments. Next, we demonstrate its practical utility in a challenging multi-objective molecular discovery setting by efficiently exploring an ultra-large synthesis-on-demand molecular library. Furthermore, we introduce a novel uncertainty quantification metric that estimates our algorithm's confidence in the predicted Pareto set. We demonstrate that this metric effectively proxies true performance, yielding a robust methodology for monitoring learning progress in complex settings. Finally, we complement these empirical findings with a theoretical proof of the algorithm's asymptotic correctness.

Problem

• Pareto Set Identification
• Multi-Objective Multi-Armed Bandits
• Bayesian Optimization
• Anytime Algorithm
• Multi-Objective Decision-Making

Innovation

• Bayesian multi-objective optimization
• anytime algorithm
• Pareto set identification
• uncertainty quantification
• multi-armed bandits