Best-of-Both-Worlds Multi-Dueling Bandits: Unified Algorithms for Stochastic and Adversarial Preferences under Condorcet and Borda Objectives

2026-03-19

AI Summary

This work addresses the challenge of unifying stochastic and adversarial preference handling in dueling bandits under unknown environments. It proposes a unified framework that requires no prior knowledge of the environment type, leveraging a MetaDueling black-box reduction to transform multi-way feedback into unbiased pairwise signals. Combined with a newly designed AlgBorda algorithm and a Versatile-DB mechanism, the approach enables adaptive learning and optimized regret bounds. Notably, it achieves the first “best-of-both-worlds” performance in multi-dueling settings: under the Condorcet objective, it simultaneously attains an adversarial pseudo-regret of $O(\sqrt{KT})$ and an instance-optimal stochastic pseudo-regret of $O(\sum \log T / \Delta_i)$; under the Borda objective, it also reaches near-optimal regret bounds. All results match existing lower bounds.

Technology Category

• Machine Learning: Online Learning & Bandits
• Game Theory and Economic Paradigms: Adversarial Learning
• Multiagent Systems: Adversarial Agents

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Fairness, privacy, and diversity in economic environments
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Multi-dueling bandits, where a learner selects $m \geq 2$ arms per round and observes only the winner, arise naturally in many applications including ranking and recommendation systems, yet a fundamental question has remained open: can a single algorithm perform optimally in both stochastic and adversarial environments, without knowing which regime it faces? We answer this affirmatively, providing the first best-of-both-worlds algorithms for multi-dueling bandits under both Condorcet and Borda objectives. For the Condorcet setting, we propose \texttt{MetaDueling}, a black-box reduction that converts any dueling bandit algorithm into a multi-dueling bandit algorithm by transforming multi-way winner feedback into an unbiased pairwise signal. Instantiating our reduction with \texttt{Versatile-DB} yields the first best-of-both-worlds algorithm for multi-dueling bandits: it achieves $O(\sqrt{KT})$ pseudo-regret against adversarial preferences and the instance-optimal $O\!\left(\sum_{i \neq a^\star} \frac{\log T}{Δ_i}\right)$ pseudo-regret under stochastic preferences, both simultaneously and without prior knowledge of the regime. For the Borda setting, we propose \AlgBorda, a stochastic-and-adversarial algorithm that achieves $O\left(K^2 \log KT + K \log^2 T + \sum_{i: Δ_i^{\mathrm{B}} > 0} \frac{K\log KT}{(Δ_i^{\mathrm{B}})^2}\right)$ regret in stochastic environments and $O\left(K \sqrt{T \log KT} + K^{1/3} T^{2/3} (\log K)^{1/3}\right)$ regret against adversaries, again without prior knowledge of the regime. We complement our upper bounds with matching lower bounds for the Condorcet setting. For the Borda setting, our upper bounds are near-optimal with respect to the lower bounds (within a factor of $K$) and match the best-known results in the literature.

Problem

• multi-dueling bandits
• best-of-both-worlds
• stochastic preferences
• adversarial preferences
• Condorcet and Borda objectives

Innovation

• best-of-both-worlds
• multi-dueling bandits
• Condorcet objective
• Borda objective
• black-box reduction