Reinforcement Learning for Machine Learning Model Deployment: Evaluating Multi-Armed Bandits in ML Ops Environments

2025-03-28

AI Summary

Manual intervention in ML Ops model deployment hinders responsiveness to model drift and sudden performance degradation. Method: We propose the first multi-armed bandit (MAB)-based reinforcement learning framework for online model evaluation, automatic model selection, and sub-second rollback—integrating UCB, ε-greedy, and Thompson Sampling strategies with real-time performance monitoring and hot-swapping mechanisms, enabling unsupervised, low-latency autonomous decision-making. Results: Experiments on two industrial-scale streaming datasets show that our RL approach improves deployment stability by 23% and reduces human intervention frequency by 91% compared to A/B testing and validation-set-based baselines; rollback latency is compressed to seconds while maintaining or exceeding baseline performance. This work provides the first systematic empirical validation of MAB efficacy in dynamic, production-grade ML Ops model management, extending reinforcement learning from model training into the end-to-end production deployment loop.

Technology Category

• Machine Learning: Online Learning & Bandits
• Humans and AI: Human-in-the-loop Machine Learning
• Multiagent Systems: Multiagent Learning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

In modern ML Ops environments, model deployment is a critical process that traditionally relies on static heuristics such as validation error comparisons and A/B testing. However, these methods require human intervention to adapt to real-world deployment challenges, such as model drift or unexpected performance degradation. We investigate whether reinforcement learning, specifically multi-armed bandit (MAB) algorithms, can dynamically manage model deployment decisions more effectively. Our approach enables more adaptive production environments by continuously evaluating deployed models and rolling back underperforming ones in real-time. We test six model selection strategies across two real-world datasets and find that RL based approaches match or exceed traditional methods in performance. Our findings suggest that reinforcement learning (RL)-based model management can improve automation, reduce reliance on manual interventions, and mitigate risks associated with post-deployment model failures.

Problem

• Dynamic model deployment using reinforcement learning
• Comparing multi-armed bandits with traditional ML Ops methods
• Reducing manual intervention in model drift scenarios

Innovation

• Reinforcement Learning manages model deployment dynamically
• Multi-armed bandit algorithms replace static heuristics effectively
• Real-time evaluation rolls back underperforming models automatically