Moving Beyond Review: Applying Language Models to Planning and Translation in Reflection

2026-03-30

AI Summary

This study addresses the common lack of depth in students’ reflective writing by proposing a novel application of large language models (LLMs) grounded in cognitive process writing theory. Moving beyond conventional LLM-based approaches that focus primarily on textual feedback, this work introduces a conversational intelligent agent designed to support learners during the early cognitive stages of reflection—specifically planning and organization. The agent employs structured prompts to guide reflective planning and automatically extracts key concepts to facilitate translation into coherent written reflections. In a controlled experiment with 93 participants, the approach significantly enhanced both the depth and structural quality of learners’ reflective writing. Behavioral and perceptual data further confirmed its effectiveness in scaffolding the reflective process, thereby advancing theory-informed AI-driven educational support.

Abstract

Reflective writing is known to support the development of students' metacognitive skills, yet learners often struggle to engage in deep reflection, limiting learning gains. Although large language models (LLMs) have been shown to improve writing skills, their use as conversational agents for reflective writing has produced mixed results and has largely focused on providing feedback on reflective texts, rather than support during planning and organizing. In this paper, inspired by the Cognitive Process Theory of writing (CPT), we propose the first application of LLMs to the planning and translation steps of reflective writing. We introduce Pensée, a tool to explore the effects of explicit AI support during these stages by scaffolding structured reflection planning using a conversational agent, and supporting translation by automatically extracting key concepts. We evaluate Pensée in a controlled between-subjects experiment (N=93), manipulating AI support across writing phases. Results show significantly greater reflection depth and structural quality when learners receive support during planning and translation stages of CPT, though these effects reduce in a delayed post-test. Analyses of learner behavior and perceptions further illustrate how CPT-aligned conversational support shapes reflection processes and learner experience, contributing empirical evidence for theory-driven uses of LLMs in AI-supported reflective writing.

Problem

• reflective writing
• large language models
• planning
• translation
• metacognition

Innovation

• large language models
• reflective writing
• Cognitive Process Theory
• planning and translation
• conversational agent