Learning from Local Walks on Dynamic Graphs with Bandit Feedback

2026-07-12

AI Summary

This work addresses the local walk-based multi-armed bandit problem on dynamic graphs, where an agent can only select actions at its current location or neighboring nodes, leading to a decoupling of exploration and exploitation. The authors introduce a process-independent sliding-window mixing condition that characterizes the stability of local connectivity under evolving graph structures. Building upon this condition, they develop a theoretically grounded local explore-and-commit algorithmic framework that integrates reward-aware policies with random walk theory. Under the proposed mixing condition, the framework achieves sublinear expected regret. Furthermore, the study establishes that the reward-aware policy guarantees both safety and performance improvement even in worst-case scenarios.

Abstract

We study stochastic multi-armed bandits on dynamic graphs, where arms correspond to the vertices of a network with time-varying edges. In this setting, the learner is restricted to local movement, selecting only its current node or an immediate neighbor at each round. This constraint decouples best-arm identification from exploitation: even after the optimal arm is identified, the learner may remain unable to reach it through the evolving topology. We identify a process-agnostic structural condition, based on sliding-window mixing, that ensures the graph's intrinsic walk remains stable for both exploration and navigation. Under this regime, we analyze a family of local explore-then-commit algorithms and establish sublinear expected regret. Our framework includes a reward-aware strategy, for which we prove a worst-case safety theorem and a separate performance gain theorem.

Problem

• dynamic graphs
• multi-armed bandits
• local walks
• bandit feedback
• best-arm identification

Innovation

• dynamic graphs
• multi-armed bandits
• local walks
• sliding-window mixing
• explore-then-commit