Adaptive Budgeted Multi-Armed Bandits for IoT with Dynamic Resource Constraints

2025-05-05

AI Summary

IoT systems operating under dynamic resource constraints—such as time-varying energy and bandwidth—struggle to simultaneously guarantee real-time performance and strict constraint satisfaction. Method: This paper proposes a budgeted multi-armed bandit (MAB) framework equipped with a novel decaying violation-budget mechanism. It introduces the first adaptive budget-decay scheme, enabling fault-tolerant exploration early in learning and progressively stricter constraint adherence later. The framework integrates adaptive Upper Confidence Bound (UCB) policies with dynamic constraint modeling to handle time-varying resource limits. Contribution/Results: Theoretically, it establishes the first sublinear regret bound and logarithmic upper bound on cumulative constraint violations under dynamic budget constraints. Empirical evaluation demonstrates a 37% improvement in constraint satisfaction rate and a 2.1× acceleration in convergence speed, empirically validating the derived theoretical guarantees.

Technology Category

• Machine Learning: Online Learning & Bandits
• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition
• Multiagent Systems: Mechanism Design

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Energy management for devices in mobile Web and WoT environments
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Internet of Things (IoT) systems increasingly operate in environments where devices must respond in real time while managing fluctuating resource constraints, including energy and bandwidth. Yet, current approaches often fall short in addressing scenarios where operational constraints evolve over time. To address these limitations, we propose a novel Budgeted Multi-Armed Bandit framework tailored for IoT applications with dynamic operational limits. Our model introduces a decaying violation budget, which permits limited constraint violations early in the learning process and gradually enforces stricter compliance over time. We present the Budgeted Upper Confidence Bound (UCB) algorithm, which adaptively balances performance optimization and compliance with time-varying constraints. We provide theoretical guarantees showing that Budgeted UCB achieves sublinear regret and logarithmic constraint violations over the learning horizon. Extensive simulations in a wireless communication setting show that our approach achieves faster adaptation and better constraint satisfaction than standard online learning methods. These results highlight the framework's potential for building adaptive, resource-aware IoT systems.

Problem

• Dynamic resource constraints in IoT systems
• Balancing performance and compliance in real-time
• Adaptive learning with time-varying operational limits

Innovation

• Budgeted Multi-Armed Bandit for dynamic IoT constraints
• Decaying violation budget for adaptive compliance
• Budgeted UCB algorithm balances performance and constraints