Compass vs Railway Tracks: Unpacking User Mental Models for Communicating Long-Horizon Work to Humans vs. AI

2026-01-17

AI Summary

This study investigates differences in users’ communication strategies and mental models when delegating long-term tasks to humans versus AI. Through qualitative analysis of task instructions authored by 16 professionals, the research reveals that users’ perception of AI’s limited capacity for intention inference and autonomous judgment leads them to adopt highly detailed, rigid “railroad-track” directives, whereas they employ higher-level, goal- and intent-oriented “compass-style” communication with human collaborators. This work is the first to identify this dual-mode strategy and argues that future AI collaborators should integrate human-like agency with machine efficiency. To this end, it proposes three design principles: supporting draft generation to align on objectives, enabling end-to-end testing to validate feasibility, and implementing intelligent checkpoints to monitor execution—thereby advancing AI from a passive executor toward a reliable collaborator.

Technology Category

• Humans and AI: Intelligent User Interfaces
• Multiagent Systems: Agent Communication
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience

Abstract

As AI systems (foundation models, agentic systems) grow increasingly capable of operating for minutes or hours at a time, users'prompts are transforming into highly detailed, elaborate specifications for the AI to autonomously work on. While interactive prompting has been extensively studied, comparatively less is known about how people communicate specifications for these types of long-horizon tasks. In a qualitative study in which 16 professionals drafted specifications for both a human colleague and an AI, we found a core divergence in how people specified problems to people versus AI: people approached communication with humans as providing a"compass", offering high-level intent to encourage flexible exploration. In contrast, communication with AI resembled painstakingly laying down"railway tracks": rigid, exhaustive instructions to minimize ambiguity and deviation. This strategy was driven by a perception that current AI has limited ability to infer intent, prioritize, and make judgments on its own. When envisioning an idealAI collaborator, users expressed a desire for a hybrid between current AI and human colleagues: a collaborator that blends AI's efficiency and large context window with the critical thinking and agency of a human colleague. We discuss design implications for future AI systems, proposing that they align on outcomes through generated rough drafts, verify feasibility via end-to-end"test runs,"and monitor execution through intelligent check-ins, ultimately transforming AI from a passive instruction-follower into a reliable collaborator for ambiguous, long-horizon problems.

Problem

• long-horizon tasks
• mental models
• human-AI collaboration
• intent communication
• autonomous AI

Innovation

• mental models
• long-horizon tasks
• AI collaboration
• human-AI communication
• autonomous agents