Bandit Optimal Transport

2025-02-11

AI Summary

This paper studies sequential optimal transport (OT) learning under known marginal distributions but unknown transportation costs. It introduces the first framework—“sequential OT learning with unknown costs”—by formulating both Kantorovich and entropy-regularized OT as stochastic multi-armed bandit problems. Through an infinite-dimensional linear bandit reduction, the approach integrates Kantorovich duality theory with RKHS regularization to design an adaptive algorithm. Leveraging the intrinsic smoothness of the cost function, the algorithm automatically tunes the regularization strength without prior knowledge, achieving an optimal Õ(√T) regret bound. It unifies treatment of both classical and entropy-regularized OT, with convergence rates that naturally interpolate according to cost smoothness. This work establishes the first theoretically grounded online learning paradigm for OT.

Technology Category

• Machine Learning: Online Learning & Bandits
• Search and Optimization: Learning to Search
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments

Abstract

Despite the impressive progress in statistical Optimal Transport (OT) in recent years, there has been little interest in the study of the emph{sequential learning} of OT. Surprisingly so, as this problem is both practically motivated and a challenging extension of existing settings such as linear bandits. This article considers (for the first time) the stochastic bandit problem of learning to solve generic Kantorovich and entropic OT problems from repeated interactions when the marginals are known but the cost is unknown. We provide $ ilde{mathcal O}(sqrt{T})$ regret algorithms for both problems by extending linear bandits on Hilbert spaces. These results provide a reduction to infinite-dimensional linear bandits. To deal with the dimension, we provide a method to exploit the intrinsic regularity of the cost to learn, yielding corresponding regret bounds which interpolate between $ ilde{mathcal O}(sqrt{T})$ and $ ilde{mathcal O}(T)$.

Problem

• Sequential learning of Optimal Transport
• Unknown cost with known marginals
• Regret algorithms for Kantorovich and entropic OT

Innovation

• Bandit Optimal Transport
• Linear bandits Hilbert spaces
• Exploit cost intrinsic regularity