🤖 AI Summary
This study investigates the short- and long-term effects of generative AI on students’ knowledge acquisition and higher-order writing skills. In a randomized controlled experiment, undergraduate participants learned a novel topic and composed analytical essays either with or without AI assistance, followed by unaided assessments immediately and one week later. Results indicate that AI support improved immediate knowledge test performance by 0.27 standard deviations, with this advantage persisting after one week. Further analysis distinguishing between “augmentation” and “automation” usage patterns revealed that augmentation—where AI complements rather than replaces student effort—significantly enhanced writing quality in subsequent unaided tasks, alongside increased information-seeking behaviors and greater reported enjoyment of learning. The findings highlight a delayed benefit of generative AI in educational contexts and underscore the critical moderating role of how such tools are employed.
📝 Abstract
We study how generative AI affects student learning in a randomized experiment. In proctored, in-person sessions, undergraduates learn about an unfamiliar topic and write an analytical essay with or without access to off-the-shelf generative AI, then complete unaided assessments immediately and one week later. We measure learning with knowledge tests (factual and conceptual understanding) and open-ended essays (higher-order skills). AI access raises immediate test scores by 0.27 standard deviations. These gains persist one week later. Essay quality, by contrast, changes little while students have AI access but improves in style and relevance one week later, when students write unaided. These delayed gains are larger among augmentation users-who use AI to explain concepts rather than generate text-whereas automation users' short-run quality gains vanish once AI is removed. We find evidence for two mechanisms behind the learning gains: students shift time away from drafting text and toward reading and searching for information, and they report greater learning enjoyment.