Modeling Distinct Human Interaction in Web Agents

2026-02-19

AI Summary

This work addresses the challenge that current networked agents struggle to accurately discern when and why human intervention occurs, often leading to erroneous autonomous decisions or excessive requests for redundant confirmations. To bridge this gap, the study formally models human intervention behavior for the first time, introduces CowCorpus—a novel dataset constructed from real user interaction trajectories—and identifies four distinct human-agent interaction patterns. Leveraging this framework, a language model is trained to predict users’ intervention intent. The proposed intervention-aware decision-making mechanism achieves a 61.4–63.4% improvement in intervention prediction accuracy over baseline methods. User studies further demonstrate a 26.5% increase in perceived agent usefulness, significantly enhancing the efficacy of human-agent collaboration.

Technology Category

• Humans and AI: Human-Aware Planning and Behavior Prediction
• Intelligent Robots: Human-Robot Interaction
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling

Abstract

Despite rapid progress in autonomous web agents, human involvement remains essential for shaping preferences and correcting agent behavior as tasks unfold. However, current agentic systems lack a principled understanding of when and why humans intervene, often proceeding autonomously past critical decision points or requesting unnecessary confirmation. In this work, we introduce the task of modeling human intervention to support collaborative web task execution. We collect CowCorpus, a dataset of 400 real-user web navigation trajectories containing over 4,200 interleaved human and agent actions. We identify four distinct patterns of user interaction with agents -- hands-off supervision, hands-on oversight, collaborative task-solving, and full user takeover. Leveraging these insights, we train language models (LMs) to anticipate when users are likely to intervene based on their interaction styles, yielding a 61.4-63.4% improvement in intervention prediction accuracy over base LMs. Finally, we deploy these intervention-aware models in live web navigation agents and evaluate them in a user study, finding a 26.5% increase in user-rated agent usefulness. Together, our results show structured modeling of human intervention leads to more adaptive, collaborative agents.

Problem

• human intervention
• web agents
• collaborative task execution
• interaction patterns
• autonomous agents

Innovation

• human intervention modeling
• web agents
• collaborative AI
• interaction patterns
• intervention prediction