Problem
Research questions and friction points this paper is trying to address.
cognitive deskilling
critical thinking
AI-assisted writing
logical reasoning
Socratic questioning
Critical Inker: Scaffolding Critical Thinking in AI-Assisted Writing Through Socratic Questioning
🤖 AI Summary
This study addresses the risk that large language model (LLM)-assisted writing may undermine users’ critical thinking by fostering cognitive offloading. To counter this, the authors propose an interactive mechanism integrating Socratic questioning with visualization of logical fallacies, designed to actively guide users in identifying and revising flaws in their arguments rather than replacing human reasoning. The system leverages an LLM to extract argumentative structures, assess logical validity, and provide dialogic feedback alongside highlighted textual cues. Experimental results show a 91.2% overlap between automatically extracted arguments and human annotations, and an 87% accuracy in logical validity judgments. Preliminary user studies demonstrate the approach’s effectiveness and feasibility in promoting critical reflection during AI-mediated writing.
📝 Abstract
As Large Language Models (LLMs) increasingly automate writing tasks, there is a growing risk of cognitive deskilling where users offload critical thinking to the system. To address this, we introduce Critical Inker, a writing tool designed to scaffold critical reflection during writing through logical analysis and socratic feedback. We present two methods: (1) A Socratic chatbot using questions to help them realize and fix logical errors in their writing and (2) Visual Feedback, which highlights logical errors in the text without dialog. We detail the technical implementation of the system and evaluate its argument extraction and logical validity accuracy. Our evaluation shows a 91.2% argument overlap with ground truth argument annotations and 87% validity accuracy. Finally, we conducted a small-scale pilot and discuss early qualitative results.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Socratic questioning
critical thinking scaffolding
logical error detection
AI-assisted writing
argument extraction
🔎 Similar Papers
No similar papers found.
