Online Convex Optimization with Dueling Feedback

2026-08-15

AI Summary

This study addresses the open problem of online optimization with pairwise comparison feedback in adversarial convex settings. We propose a novel reduction that transforms pairwise feedback into approximate gradients, enabling the application of standard first-order algorithms. Through rigorous regret analysis, we establish static, adaptive, and dynamic regret bounds. Specifically, this work achieves a general regret bound of o(t^{3/4}), which improves to o(t^{2/3}) under smoothness assumptions and o(√(t log t)) under strong convexity. By bridging this theoretical gap, our research provides the first comprehensive theoretical guarantees and an effective algorithmic framework for adversarial online learning relying solely on ordinal feedback.

Abstract

We study online convex optimization with dueling (pairwise comparison) feedback, where the learner observes only a binary preference between two queried points. While dueling feedback is well understood in discrete or stochastic settings, the adversarial convex setting has remained unexplored. We propose a simple reduction that converts dueling feedback into approximate gradients, enabling the use of standard first-order methods. We show that regret guarantees transfer under this reduction, yielding the first results for this setting, including $\mathcal{O}(T^{3/4})$ static, adaptive, and dynamic regret. Under additional structure, we obtain improved rates of $\mathcal{O}(T^{2/3})$ for smooth objectives and $\mathcal{O}(\sqrt{T \log T})$ for strongly convex functions.

Problem

• Online Convex Optimization
• Dueling Feedback
• Adversarial Convex Setting
• Regret Bounds

Innovation

• Online Convex Optimization
• Dueling Feedback
• Gradient Approximation
• Regret Bounds
• Adversarial Setting