Bandit PCA with Minimax Optimal Regret

2026-07-12

AI Summary

This work studies online principal component analysis (PCA) with bandit feedback, where the learner sequentially selects unit vectors to maximize cumulative reward against an unknown sequence of symmetric low-rank gain matrices. The authors propose a novel algorithm that combines online mirror descent over the spectral simplex with multi-scale exploration of eigensubspaces. By constructing an adaptive adversarial setting, they reduce the regret lower bound to a subspace estimation problem and establish, for the first time, a minimax regret lower bound of $\Omega(r\sqrt{dT})$ for this setting. Their algorithm achieves a matching upper bound of $O(r\sqrt{dT})$, up to polylogarithmic factors, thereby characterizing the minimax optimal regret rate for bandit PCA.

Abstract

We study the bandit-feedback version of online principal component analysis (Bandit PCA): in each round $t = 1,\dots,T$, the adversary selects a $d \times d$ symmetric gain matrix $G_t$ with spectrum in $[0,1]$ and rank at most $r$; the learner simultaneously selects a unit vector $w_t \in S^{d-1}$ and receives the reward $w_t^\top G_t w_t$. The learner receives no other feedback, and aims to minimize the regret against the best unit vector in hindsight. This problem was introduced by Kotlowski and Neu (2019), who gave an algorithm with regret $O(d\sqrt{rT \log T})$ and showed the lower bound of $Ω(r\sqrt{T/\log T})$. We improve upon both of these bounds and essentially bridge the gap between them, establishing the minimax regret of order $r\sqrt{dT}$ up to polylogarithmic factors in $d$ and $T$. The upper bound is attained by a novel algorithm, which combines online mirror descent on the spectrahedron of (real) density matrices with a multiscale exploration scheme in which the eigenspaces with different spectral magnitudes are updated at different rates. For the lower bound, we construct an adaptive adversary that refines a hidden large-reward subspace based on the learner's actions, in such a way that low regret is impossible without estimating the subspace; as a result, lower-bounding the regret reduces to studying the arising subspace estimation problem. Finally, we discuss connections of Bandit PCA with adaptive-measurement quantum tomography.

Problem

• Bandit PCA
• online principal component analysis
• minimax regret
• bandit feedback
• adaptive adversary

Innovation

• Bandit PCA
• minimax regret
• online mirror descent
• multiscale exploration
• adaptive adversary