When not to help: planning for lasting human-AI collaboration

2025-08-03

AI Summary

In real-time AI-human interaction, determining “when to assist” remains a critical challenge—excessive intervention reduces user engagement and undermines long-term collaborative efficacy. Method: We propose a POMDP-based cognitive modeling framework that integrates Bayesian inference with reinforcement learning to dynamically estimate users’ latent cognitive states. It explicitly incorporates long-term collaborative sustainability—balancing task performance and user engagement—as a core decision objective, and introduces counterfactual reasoning to predict users’ unassisted performance, thereby quantifying the trade-off between assistance benefits and engagement loss risk. Contribution/Results: Simulation results demonstrate that our adaptive assistance policy significantly outperforms baseline strategies—namely, always-assist and never-assist—in both task completion quality and sustained user engagement, establishing a principled foundation for sustainable human-AI collaboration.

Technology Category

• Humans and AI: Planning and Decision Support for Human-Machine Teams
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior
• Intelligent Robots: Human-Robot Interaction

Application Category

• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Research challenges in human and human-AI computation

Abstract

AI systems and technologies that can interact with humans in real time face a communication dilemma: when to offer assistance and how frequently. Overly frequent or contextually redundant assistance can cause users to disengage, undermining the long-term benefits of AI assistance. We introduce a cognitive modeling framework based on Partially Observable Markov Decision Processes (POMDPs) that addresses this timing challenge by inferring a user's latent cognitive state related to AI engagement over time. Additionally, our framework incorporates reasoning about the long-term effects of AI assistance, explicitly aiming to avoid actions that could lead the human user to disengage or deactivate the AI. A key component of our approach is counterfactual reasoning: at each time step, the AI considers how well the user would perform independently and weighs the potential boost in performance against the risk of diminishing engagement with the AI. Through simulations, we show that this adaptive strategy significantly outperforms baseline policies in which assistance is always provided or never provided. Our results highlight the importance of balancing short-term decision accuracy with sustained user engagement, showing how communication strategies can be optimized to avoid alert fatigue while preserving the user's receptiveness to AI guidance.

Problem

• Determining optimal timing for AI assistance to prevent user disengagement
• Modeling user cognitive states to balance short-term and long-term AI benefits
• Avoiding alert fatigue while maintaining user receptiveness to AI guidance

Innovation

• POMDPs model user's latent cognitive state
• Counterfactual reasoning balances performance and engagement
• Adaptive strategy prevents user disengagement effectively