An Experimental Comparison of Cognitive Forcing Functions for Execution Plans in AI-Assisted Writing: Effects On Trust, Overreliance, and Perceived Critical Thinking

2026-01-25

AI Summary

This study addresses the risk of overreliance on generative AI in knowledge-intensive tasks such as writing, which can undermine users’ critical thinking—particularly when AI provides execution plans. It presents the first systematic investigation into the role of cognitive forcing functions in this context, evaluating four conditions—Assumption prompts, WhatIf prompts, their combination, and no intervention—through controlled experiments, think-aloud protocols, and semi-structured interviews. The findings reveal that Assumption prompts most effectively reduce overreliance without increasing cognitive load, whereas WhatIf prompts are perceived by users as most helpful. These results demonstrate the value of plan-oriented cognitive forcing functions in fostering critical reflection and highlight a trade-off between mitigating dependence and maintaining manageable cognitive demands across different intervention strategies.

Technology Category

• Humans and AI: Planning and Decision Support for Human-Machine Teams
• Reasoning under Uncertainty: Causality
• Cognitive Modeling & Cognitive Systems: Computational Creativity

Application Category

• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Generative AI (GenAI) tools improve productivity in knowledge workflows such as writing, but also risk overreliance and reduced critical thinking. Cognitive forcing functions (CFFs) mitigate these risks by requiring active engagement with AI output. As GenAI workflows grow more complex, systems increasingly present execution plans for user review. However, these plans are themselves AI-generated and prone to overreliance, and the effectiveness of applying CFFs to AI plans remains underexplored. We conduct a controlled experiment in which participants completed AI-assisted writing tasks while reviewing AI-generated plans under four CFF conditions: Assumption (argument analysis), WhatIf (hypothesis testing), Both, and a no-CFF control. A follow-up think-aloud and interview study qualitatively compared these conditions. Results show that the Assumption CFF most effectively reduced overreliance without increasing cognitive load, while participants perceived the WhatIf CFF as most helpful. These findings highlight the value of plan-focused CFFs for supporting critical reflection in GenAI-assisted knowledge work.

Problem

• generative AI
• overreliance
• critical thinking
• execution plans
• cognitive forcing functions

Innovation

• cognitive forcing functions
• execution plans
• generative AI
• overreliance
• critical thinking