Constrained Pareto Set Identification with Bandit Feedback

2025-06-09

AI Summary

This paper studies Pareto-optimal arm identification in multi-objective stochastic bandits under linear feasibility constraints: given unknown mean vectors μ₁,…,μₖ ∈ ℝᵈ, the goal is to identify all arms satisfying the constraints and undominated in the Pareto sense. We propose the first single-stage adaptive algorithm that jointly integrates multi-objective confidence interval construction, constraint feasibility testing, and an optimal stopping rule. Theoretically, we establish that its sample complexity matches the information-theoretic lower bound up to constant factors—significantly improving upon conventional two-phase and racing-style methods. Extensive experiments across multiple benchmarks demonstrate superior sampling efficiency, faster convergence, and enhanced robustness. Our work provides both tight theoretical guarantees and a practical, efficient solution for constrained multi-objective sequential decision-making.

Technology Category

• Constraint Satisfaction and Optimization: Constraint Optimization
• Reasoning under Uncertainty: Stochastic Optimization
• Machine Learning: Online Learning & Bandits

Application Category

• Economics, Online Markets and Human Computation: Fairness, privacy, and diversity in economic environments
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

In this paper, we address the problem of identifying the Pareto Set under feasibility constraints in a multivariate bandit setting. Specifically, given a $K$-armed bandit with unknown means $mu_1, dots, mu_K in mathbb{R}^d$, the goal is to identify the set of arms whose mean is not uniformly worse than that of another arm (i.e., not smaller for all objectives), while satisfying some known set of linear constraints, expressing, for example, some minimal performance on each objective. Our focus lies in fixed-confidence identification, for which we introduce an algorithm that significantly outperforms racing-like algorithms and the intuitive two-stage approach that first identifies feasible arms and then their Pareto Set. We further prove an information-theoretic lower bound on the sample complexity of any algorithm for constrained Pareto Set identification, showing that the sample complexity of our approach is near-optimal. Our theoretical results are supported by an extensive empirical evaluation on a series of benchmarks.

Problem

• Identify Pareto Set under feasibility constraints in bandits
• Optimize sample complexity for constrained multi-objective bandits
• Prove near-optimality of proposed algorithm via lower bounds

Innovation

• Bandit algorithm for constrained Pareto Set
• Near-optimal sample complexity proven
• Outperforms racing and two-stage approaches