Optimism as a Vulnerability: Deceptive Stackelberg Control of UCB Bandit Followers

2026-06-28

AI Summary

This work addresses the inadequacy of traditional Stackelberg equilibria when facing a boundedly rational follower employing the Upper Confidence Bound (UCB) algorithm for experiential learning, as the follower’s optimistic exploration can be strategically exploited. The authors propose a two-stage deception mechanism: in an initial baiting phase, the leader artificially inflates the UCB index of a target action; subsequently, in a trapping phase, the leader switches to a self-interested policy, inducing the follower—relying on the manipulated history—to persistently select that action. This approach constitutes the first provably effective deceptive mechanism for leaders confronting UCB followers, revealing a fundamental incompatibility between static equilibrium concepts and dynamic learning incentives. Under standard separation and payoff assumptions, the leader’s cumulative utility strictly exceeds the classical Stackelberg equilibrium upper bound, with manipulation cost bounded within an $O(\sqrt{T \ln T})$ regret term.

Abstract

Upper Confidence Bound (UCB) algorithms guarantee sublinear regret for agents learning unknown stochastic environments, yet the same principle that makes them statistically efficient (optimism in the face of uncertainty) induces a predictable strategic vulnerability against an omniscient adaptive leader. Classical strong Stackelberg equilibrium (SSE) assumes that the follower immediately best-responds to the leader's committed mixed action; it therefore supplies no mechanism-design prescription for a leader facing a boundedly rational follower who constructs and acts on empirical reward histories. We formalize this conflict in a finite-horizon repeated Stackelberg game and give exact constructive proofs for a deceptive leader mechanism. In a honeypot phase, the leader pays a finite signaling cost to inflate the UCB index of a designated follower action. In a trap phase, the leader switches to a selfish action distribution while the follower remains locked into the designated action because the manipulated empirical history and exploration bonus dominate competing indices. Under explicit separation and payoff assumptions, the leader's cumulative utility strictly exceeds the classical SSE ceiling, and the manipulation cost is bounded by a regret calculation of order $O(\sqrt{T\ln T})$. The results identify a formal incompatibility between static equilibrium prescriptions and dynamically learned empirical incentives.

Problem

• Stackelberg equilibrium
• UCB bandits
• optimism
• deception
• bounded rationality

Innovation

• Deceptive Stackelberg Control
• UCB Bandits
• Optimism Vulnerability
• Strategic Manipulation
• Empirical Incentives