Better Bounds for the Distributed Experts Problem

2026-03-10

AI Summary

This work addresses the problem of distributed online learning with experts, where $n$ experts are distributed across $s$ servers and the loss at each round is given by the $\ell_p$ norm of the servers’ loss vectors. The authors propose a novel protocol that integrates randomized sampling with sparse communication to achieve, for the first time, a near-optimal trade-off between regret and communication under $\ell_p$-norm losses. By exploiting the structure of the $\ell_p$ norm and optimizing logarithmic factors, the algorithm attains a cumulative regret bound of $R \approx 1/(\sqrt{T} \cdot \mathrm{polylog}(nsT))$ while using only $O\big((n + s)/R^2 \cdot \max(s^{1-2/p}, 1) \cdot \mathrm{polylog}(nsT)\big)$ bits of communication. This significantly improves upon existing methods in both regret and communication efficiency.

Technology Category

• Machine Learning: Online Learning & Bandits
• Search and Optimization: Distributed Search
• Multiagent Systems: Agent Communication

Application Category

• Economics, Online Markets and Human Computation: Social networks and social learning
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

In this paper, we study the distributed experts problem, where $n$ experts are distributed across $s$ servers for $T$ timesteps. The loss of each expert at each time $t$ is the $\ell_p$ norm of the vector that consists of the losses of the expert at each of the $s$ servers at time $t$. The goal is to minimize the regret $R$, i.e., the loss of the distributed protocol compared to the loss of the best expert, amortized over the all $T$ times, while using the minimum amount of communication. We give a protocol that achieves regret roughly $R\gtrsim\frac{1}{\sqrt{T}\cdot\text{poly}\log(nsT)}$, using $\mathcal{O}\left(\frac{n}{R^2}+\frac{s}{R^2}\right)\cdot\max(s^{1-2/p},1)\cdot\text{poly}\log(nsT)$ bits of communication, which improves on previous work.

Problem

• distributed experts
• regret minimization
• communication complexity
• online learning
• ℓ_p norm

Innovation

• distributed experts
• regret minimization
• communication complexity
• ℓ_p norm
• online learning