Hybrid Combinatorial Multi-armed Bandits with Probabilistically Triggered Arms

2025-12-26

AI Summary

In combinatorial multi-armed bandits with probabilistically triggered arms (CMAB-T), existing online methods suffer from high interaction costs and slow adaptation, while offline methods are hindered by data quality limitations and insufficient exploration capability. Method: We propose the first hybrid CMAB-T theoretical framework and design Hybrid CUCB—a novel algorithm that synergistically integrates offline-data-guided efficient exploration with online-interaction-driven bias correction. Contribution/Results: We establish a provably improved regret bound. Theoretically, under high-quality offline data, the cumulative regret is significantly reduced. Empirically, Hybrid CUCB achieves 37%–62% faster convergence compared to purely online or purely offline baselines, while demonstrating superior robustness and generalization across diverse problem instances.

Technology Category

• Machine Learning: Online Learning & Bandits
• Search and Optimization: Mixed Discrete/Continuous Search
• Game Theory and Economic Paradigms: Adversarial Learning

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: User modeling for targeted and personalized online advertising

Abstract

The problem of combinatorial multi-armed bandits with probabilistically triggered arms (CMAB-T) has been extensively studied. Prior work primarily focuses on either the online setting where an agent learns about the unknown environment through iterative interactions, or the offline setting where a policy is learned solely from logged data. However, each of these paradigms has inherent limitations: online algorithms suffer from high interaction costs and slow adaptation, while offline methods are constrained by dataset quality and lack of exploration capabilities. To address these complementary weaknesses, we propose hybrid CMAB-T, a new framework that integrates offline data with online interaction in a principled manner. Our proposed hybrid CUCB algorithm leverages offline data to guide exploration and accelerate convergence, while strategically incorporating online interactions to mitigate the insufficient coverage or distributional bias of the offline dataset. We provide theoretical guarantees on the algorithm's regret, demonstrating that hybrid CUCB significantly outperforms purely online approaches when high-quality offline data is available, and effectively corrects the bias inherent in offline-only methods when the data is limited or misaligned. Empirical results further demonstrate the consistent advantage of our algorithm.

Problem

• Hybrid CMAB-T integrates offline data with online interaction to overcome individual limitations.
• It leverages offline data to guide exploration and accelerate convergence in bandit problems.
• The approach corrects bias in offline-only methods and reduces interaction costs in online algorithms.

Innovation

• Integrates offline data with online interaction
• Uses offline data to guide exploration and accelerate convergence
• Incorporates online interactions to correct offline dataset bias