Sequential learning of the Pareto front for multi-objective bandits

2025-01-29

International Conference on Artificial Intelligence and Statistics

AI Summary

This paper addresses the sequential identification of the Pareto front in multi-objective multi-armed bandits, aiming to exactly identify all Pareto-optimal arms with probability at least $1-delta$. We propose the first computationally efficient algorithm achieving the theoretical optimal sample complexity. Its per-round time complexity is $O(Kp^d)$, significantly improving upon existing methods. The core innovations include: (i) adaptive construction of confidence intervals tailored to multi-objective rewards; (ii) dynamic判定 of Pareto dominance relations; and (iii) an incremental sampling strategy that jointly optimizes statistical accuracy and computational efficiency. We prove that the algorithm achieves the information-theoretic lower bound asymptotically as $delta o 0$. Empirical evaluation demonstrates superior performance in high-dimensional multi-objective settings—achieving higher identification accuracy and faster convergence compared to state-of-the-art baselines.

Technology Category

• Machine Learning: Online Learning & Bandits
• Search and Optimization: Sampling/Simulation-based Search
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Economics, Online Markets and Human Computation: Fairness, privacy, and diversity in economic environments
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

We study the problem of sequential learning of the Pareto front in multi-objective multi-armed bandits. An agent is faced with K possible arms to pull. At each turn she picks one, and receives a vector-valued reward. When she thinks she has enough information to identify the Pareto front of the different arm means, she stops the game and gives an answer. We are interested in designing algorithms such that the answer given is correct with probability at least 1-$delta$. Our main contribution is an efficient implementation of an algorithm achieving the optimal sample complexity when the risk $delta$ is small. With K arms in d dimensions p of which are in the Pareto set, the algorithm runs in time O(Kp^d) per round.

Problem

• Multi-objective Decision Making
• Pareto Optimal Solutions
• Probability of Correctness

Innovation

• Multi-objective Decision-making
• Pareto Optimal Solutions
• Efficient Algorithm