Graph Dimensionality Reduction for Contextual Bandits: Structure-Specific Regret Bounds under Approximate Smoothness and Noisy Eigenspaces

2026-06-26

AI Summary

This work addresses the inefficiency of conventional dimensionality reduction in contextual bandits with graph-structured arms, where ignoring graph information leads to exploration complexity scaling with the ambient dimension $d$ rather than the effective dimension $k$. The authors propose projecting arm features onto the low-frequency subspace of the graph Laplacian and running linear UCB in this $k$-dimensional space, establishing the first $\tilde{O}(k\sqrt{T})$ regret bound for spectral projection-based bandits. The analysis reveals that the practical cost of high-frequency reward components depends on their influence along the policy’s trajectory rather than their total energy. A threshold-free subspace spectral comparison criterion is introduced to predict algorithmic performance. Experiments on six real-world datasets show a 15-fold average reduction in cumulative regret, outperforming existing graph-aware methods in five cases, with failures precisely aligning with mismatches between the graph spectral subspace and the reward signal.

Abstract

Contextual bandits with graph-structured arms arise in recommendation, citation retrieval, and social advertising, where arms connected on a graph tend to share reward signal. Standard dimensionality reduction ignores this structure, inflating exploration cost by a factor of $d/k$. We propose GraphDR-LinUCB, which projects arm features onto the graph's low-frequency spectral subspace and runs linear UCB in the resulting $k$-dimensional space. We prove the first $\wtO(k\sqrt{T})$ regret bound for spectral-projection-based contextual bandits, reducing dimension dependence from $d$ to $k$; a perturbation argument extends this to noisy graphs, with an explicit penalty for reward-smoothness mismatch and graph-estimation error. Our central theoretical finding is that the high-frequency reward component need not incur a worst-case linear-in-$T$ penalty: its actual cost depends on its realized impact along the played path, not on its total energy. A simple spectral comparison between subspaces ($Γ_k$) predicts which reducer wins on a given dataset, correctly calling five of six real-dataset outcomes without any fitted threshold. Across a synthetic benchmark and six real datasets (MovieLens, Amazon, LastFM, ogbn-arxiv, MIND), GraphDR-LinUCB reduces cumulative regret by $15\times$ over full-dimensional LinUCB and outperforms competing graph-aware methods on five of six; the single failure is precisely where the graph's spectral subspace is misaligned with the reward.

Problem

• contextual bandits
• graph-structured arms
• dimensionality reduction
• regret minimization
• spectral subspace

Innovation

• Graph Dimensionality Reduction
• Contextual Bandits
• Spectral Subspace
• Regret Bound
• Noisy Graphs