Latent Order Bandits

2026-05-08

AI Summary

This work addresses the limitations of traditional latent-state bandits, which require exact knowledge of the joint distribution of rewards and latent states and assume identical reward distributions across homogeneous instances—assumptions that often fail in real-world settings with vastly differing reward scales. To overcome this, the paper proposes the Latent-Ordinal Bandit (LOB), which only requires a priori knowledge of partial orderings over action preferences within each latent state. LOB allows instances sharing the same latent state to exhibit distinct reward distributions while preserving consistent preference structures. This approach introduces, for the first time, a partial-order framework to model cross-instance commonality, relaxing strong prior assumptions without sacrificing flexibility or learnability. By integrating UCB and posterior sampling strategies, LOB matches the performance of full-prior methods under shared parameters and significantly outperforms existing models when reward scales vary, demonstrating superior robustness and sample efficiency.

Abstract

Bandit algorithms solve diverse sequential decision-making problems, but are often too sample-inefficient for from-scratch personalization. To substantially reduce exploration times, latent bandit algorithms exploit cross-instance structure implied by discrete latent states, provided that the posterior distribution of rewards and latent states is known and accurate. However, obtaining an accurate model of this structure is difficult, and a small number of latent states may be insufficient to characterize the reward distributions in all problem instances. We propose latent order bandits (LOB), relaxing the assumptions of latent bandits to require only prior knowledge of a partial order of action preferences in each state. This allows instances of the same state to vary in reward distributions, as long as the partial order of actions is shared. For example, groups of users on a streaming service may agree on which movie genres are the best but rate experiences on different scales. We give an upper-confidence bound procedure for the LOB problem, applicable to both total and partial latent orders, and give an upper bound on its regret. To improve empirical performance, we propose a posterior-sampling algorithm and show, in a suite of experiments, that both are competitive with full-prior latent bandits when same-state instances share reward parameters, and preferable to them when reward scales differ between instances with the same latent state.

Problem

• latent bandits
• sequential decision-making
• reward distribution
• partial order
• personalization

Innovation

• Latent Order Bandits
• partial order
• sample efficiency
• posterior sampling
• regret bound