Selective Ensemble Based on Preference-Directed Multi-Objective Bandits

2026-06-20

AI Summary

This work addresses model selection under budget constraints when downstream tasks provide only partial preference information—such as accuracy or robustness—by formulating it as a Preference-guided Dynamic Multi-objective Bandit (PDMOB) problem. The authors introduce a polyhedral preference cone to characterize acceptable trade-offs and propose the PrefUCB algorithm, which leverages directional confidence intervals to guide exploration. Key contributions include the notion of Pareto C-optimality, which unifies classical Pareto optimality and single-weight scalarization, and the first instance-dependent logarithmic regret bounds for both indicator-based and gap-weighted regret metrics. Empirical validation on large-model ensembles and constrained online portfolio allocation demonstrates the method’s efficacy, while theoretical analysis confirms that it achieves optimal logarithmic regret in classical settings.

Abstract

Selective ensemble for modern machine learning systems requires choosing promising model candidates under limited evaluation budgets, while downstream tasks often specify only partial preferences over capabilities such as accuracy, robustness, and reasoning. This setting naturally gives rise to a sequential decision problem under partially specified linear preferences. We formalize it as preference-directed multi-objective bandits (PDMOB), where admissible trade-offs are represented by a polyhedral preference cone. Based on this formulation, we introduce Pareto $C$-optimality, which recovers standard Pareto optimality and single-weight scalarization as special cases. We then propose the preference-directed upper confidence bound (PrefUCB) algorithm, which maintains directional confidence intervals to guide exploration. We analyze both indicator-based and gap-weighted regret, and establish instance-dependent logarithmic bounds for both criteria, recovering the optimal logarithmic dependence on the horizon $T$ in classical special cases. Experiments on large pre-trained model selective ensemble tasks and online asset allocation under institutional mandates validate the efficacy of our method.

Problem

• selective ensemble
• partial preferences
• multi-objective bandits
• model selection
• preference-directed

Innovation

• preference-directed multi-objective bandits
• selective ensemble
• Pareto C-optimality
• PrefUCB
• polyhedral preference cone