Learning When to Automate: Queue Control in Human-AI Service Systems

2026-07-07

AI Summary

This study addresses the dynamic decision-making problem of task automation levels in human–AI collaborative service systems, focusing on a two-stage hybrid architecture where tasks are first handled by a chatbot and escalated to human agents when necessary. Under uncertainty regarding task-type success probabilities and human service rates, the work balances robotic operational costs against human queue congestion. The authors innovatively integrate online learning with queue stability control, proposing a novel decision-making paradigm that jointly accounts for parameter learning and queue-awareness. They design the UCB-DPP policy, which combines Upper Confidence Bounds with the Drift-Plus-Penalty method. Theoretical analysis establishes an $\tilde{O}(K\sqrt{T})$ regret bound while guaranteeing mean-rate stability of the human queue, and simulations demonstrate significant performance improvements over existing baselines.

Abstract

We study a human-AI service system in which tasks arrive sequentially and are processed through a two-stage architecture: an automated chatbot followed, when necessary, by a human agent. We consider $T$ sequentially arriving tasks, each belonging to one of $K$ heterogeneous types. For each task the decision maker chooses how many resources to allocate to the chatbot, whose type-dependent success probabilities are initially unknown. Tasks not resolved by the chatbot enter type-dependent human-service queues, where they are processed by a human agent with unknown service rates. This model captures a central tradeoff in hybrid service systems: relying more on automation reduces human congestion but increases chatbot costs, while insufficient automation may overload the human agent. We propose the UCB-DPP policy, which combines Upper Confidence Bounds with Drift-Plus-Penalty control to learn the unknown parameters of the system while making queue-aware decisions. We prove that UCB-DPP achieves regret $\widetilde{\mathcal{O}}(K\sqrt{T})$ and guarantees mean-rate stability of the human-service queues. Simulations on synthetic instances show that the proposed policy outperforms natural baselines.

Problem

• human-AI service systems
• queue control
• automation decision
• resource allocation
• hybrid service systems

Innovation

• Human-AI service systems
• queue control
• Upper Confidence Bound
• Drift-Plus-Penalty
• online learning