Generalized Linear Bandits with Memory

2026-08-16

AI Summary

This study addresses the challenge of non-stationary decision-making in generalized linear multi-armed bandits with memory effects. We propose a block-based algorithm utilizing shrinking confidence bounds to jointly handle nonlinear rewards and memory-induced non-stationarity. Through refined theoretical analysis, we decouple the dominant regret term from the curvature of the link function. Consequently, this method achieves an optimal O(√T) regret rate for both linear and generalized linear settings, with empirical results validating our theoretical findings. The core contribution lies in overcoming limitations of traditional analytical frameworks, thereby providing a unified solution with optimal rate guarantees for non-stationary generalized linear bandits.

Abstract

We study generalized linear bandits with memory, an endogenous non-stationary setting in which rewards depend on past actions through a finite memory matrix. Building on prior work for linear models (Clerici et al., 2024), we show that the previously known $\tilde{O}(T^{3/4})$ regret bound stems from a loose analysis, and we provide a sharpened analysis that recovers a $\tilde{O}(\sqrt{T})$ regret rate in the linear case. We then extend this improvement to generalized linear models and propose a block-wise algorithm based on shrunken confidence bounds. Our algorithm achieves a regret bound of $\tilde{O}\left(\sqrt{mT} + d\sqrt{T} + \sqrtκ\, d^{2} m^{1/4} T^{1/4} + κd^{2} \right)$, where $d$ denotes the feature dimension, $m$ the memory length, and $κ$ a curvature parameter of the link function. This attains a $\sqrt{T}$-type rate despite nonlinear rewards and memory effects. To the best of our knowledge, this analysis provides a unified treatment of memory-induced non-stationarity and nonlinear link functions, while ensuring that the leading regret term is independent of the curvature of the link function. We conduct numerical experiments that are consistent with our theoretical findings.

Problem

• Generalized Linear Bandits
• Memory Effects
• Endogenous Non-stationarity
• Regret Analysis
• Nonlinear Rewards

Innovation

• Generalized Linear Bandits with Memory
• Shrunken Confidence Bounds
• Block-wise Algorithm
• Regret Analysis
• Non-stationary Setting