Problem
Research questions and friction points this paper is trying to address.
metacognitive demand
AI assistance
intervention design
user competence
system competence
Beyond "ChatGPT Can Make Mistakes": Designing Interventions to Support Metacognitive Monitoring in AI-Assisted Work
🤖 AI Summary
研究设计了干预措施以支持AI辅助工作中的元认知监控，通过实验比较不同方法效果，发现可靠性卡片和对比回复能减少估计误差和过度自信。
📝 Abstract
AI assistance places a metacognitive demand on users, who must judge their own competence and the system's. Yet designers lack comparative evidence on which interventions to choose, where to place them, and how to tell whether they worked. We elicited 30 interventions from 11 experts and, with prior work, organized them into a design space of time (when an intervention acts), level (whose competence is judged), and source (who supplies the monitoring cue). A between-subjects experiment (N = 917; 12 planning-and-organizing problems) compared a per-task reliability card, contrasting replies, pause points, and post-problem reflection against a baseline LLM assistant. Reliability cards and contrasting replies reduced estimation error and overconfidence and increased aggregate confidence discrimination. No task-performance improvement or average within-item discrimination gain was established. We contribute a shared vocabulary, a design space, and evidence that measured monitoring and task performance are separable design targets.
Innovation
Methods, ideas, or system contributions that make the work stand out.
metacognitive monitoring
design space
intervention design
reliability card
contrasting replies
🔎 Similar Papers
No similar papers found.
