On the Universal Near Optimality of Hedge in Combinatorial Settings

2025-10-19

AI Summary

This paper investigates the universal near-optimality of the Hedge algorithm in combinatorial online learning. Methodologically, it employs adversarial lower bound constructions, analysis within the Online Mirror Descent (OMD) framework, and dilated entropy regularization. The main contributions are threefold: (i) For arbitrary combinatorial action sets (mathcal{X}), Hedge achieves regret (O(sqrt{T log|mathcal{X}| / log d})), matching the information-theoretic lower bound (Omega(sqrt{T log|mathcal{X}| / log d})) up to a (sqrt{log d}) factor—establishing its universal approximate optimality; (ii) Hedge is provably suboptimal on structured domains such as (m)-subsets; (iii) In online multitask learning and the DAG shortest-path problem, Hedge attains exact minimax-optimal and nearly tight regret bounds, respectively, and is shown to be iterationally equivalent to OMD over path polytopes.

Technology Category

• Machine Learning: Online Learning & Bandits
• Search and Optimization: Learning to Search
• Game Theory and Economic Paradigms: Adversarial Learning

Application Category

• Economics, Online Markets and Human Computation: Social networks and social learning
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

In this paper, we study the classical Hedge algorithm in combinatorial settings. In each round, the learner selects a vector $oldsymbol{x}_t$ from a set $X subseteq {0,1}^d$, observes a full loss vector $oldsymbol{y}_t in mathbb{R}^d$, and incurs a loss $langle oldsymbol{x}_t, oldsymbol{y}_t angle in [-1,1]$. This setting captures several important problems, including extensive-form games, resource allocation, $m$-sets, online multitask learning, and shortest-path problems on directed acyclic graphs (DAGs). It is well known that Hedge achieves a regret of $Oig(sqrt{T log |X|}ig)$ after $T$ rounds of interaction. In this paper, we ask whether Hedge is optimal across all combinatorial settings. To that end, we show that for any $X subseteq {0,1}^d$, Hedge is near-optimal--specifically, up to a $sqrt{log d}$ factor--by establishing a lower bound of $Ωig(sqrt{T log(|X|)/log d}ig)$ that holds for any algorithm. We then identify a natural class of combinatorial sets--namely, $m$-sets with $log d leq m leq sqrt{d}$--for which this lower bound is tight, and for which Hedge is provably suboptimal by a factor of exactly $sqrt{log d}$. At the same time, we show that Hedge is optimal for online multitask learning, a generalization of the classical $K$-experts problem. Finally, we leverage the near-optimality of Hedge to establish the existence of a near-optimal regularizer for online shortest-path problems in DAGs--a setting that subsumes a broad range of combinatorial domains. Specifically, we show that the classical Online Mirror Descent (OMD) algorithm, when instantiated with the dilated entropy regularizer, is iterate-equivalent to Hedge, and therefore inherits its near-optimal regret guarantees for DAGs.

Problem

• Analyzing Hedge algorithm's optimality in combinatorial online learning settings
• Establishing regret lower bounds for combinatorial decision sets
• Identifying settings where Hedge is provably suboptimal or optimal

Innovation

• Hedge algorithm achieves near-optimal combinatorial regret
• Established tight lower bounds for all combinatorial algorithms
• Online Mirror Descent with dilated entropy equals Hedge