A Simple Solution to Improving Human Supervision of Algorithms: Evidence from Smart Vending

2026-07-01

AI Summary

This study addresses the challenge of preserving the value of human private information in AI decision-making while mitigating the detrimental effects of human bias and noise. The authors propose a constrained human-in-the-loop intervention mechanism that automatically filters high-value interventions by limiting the frequency of human input within each decision cycle, without requiring algorithmic redesign, customized information structures, or additional training. Leveraging a randomized field experiment and Local Average Treatment Effect (LATE) analysis, the findings show that constrained intervention reduces inventory by 1.28% without compromising sales, whereas unconstrained intervention leads to a 1.19% decline in sales. Moreover, integrating personalized strategies with the constrained approach increases the probability of a sale by 9.1%. This framework offers a lightweight, efficient, and scalable pathway for optimizing human–AI collaborative decision-making.

Abstract

Organizations increasingly deploy autonomous artificial intelligence (AI) systems for operational decisions, such as inventory replenishment. Yet fully granting override rights can degrade performance due to human bias and noise, while prohibiting them may overlook valuable private information. This raises a key question: How should override rights be structured to improve human supervision of autonomous AI? Methodology/results: We propose a constrained override policy that limits overrides per decision episode to enable selective filtering that prioritizes high-value overrides. We tested it through a randomized field experiment with 553 workers at a major Chinese smart vending machine retailer that manages more than 59,000 machines and 4,000 SKUs. Workers were assigned to no overrides, free overrides, or a two-per-machine limit on downward overrides. Free overrides reduce inventory by 1.95% but also cut sales by 1.19%. Constrained overrides reduce inventory by 1.28% without harming sales, as workers select better SKUs to override, confirmed via local average treatment effects. Gains are largest for experienced workers, high-incentive SKUs, and growth-stage SKUs. A simulated personalized policy further increases sales probability by 9.1%. Managerial implications: Academics gain novel insights from the causal effects of discretion design in human-supervised AI, emphasizing selective filtering to enhance decision quality. Managers can benefit from a scalable, low-cost policy for operations such as retail, logistics, and resource planning, reducing excess inventory without sales loss while harnessing private human information, with no need for algorithmic redesign, information customization, or additional training.

Problem

• human supervision
• AI override rights
• autonomous AI
• decision discretion
• human-AI collaboration

Innovation

• constrained override
• human-AI collaboration
• selective filtering
• inventory optimization
• field experiment