Learning from examples: AI assistance can enhance rather than hinder skill development

2025-02-05

AI Summary

This study investigates whether AI writing tools enhance—rather than impair—long-term human writing competence. Method: Three randomized controlled trials (RCTs) compared the effects of AI-assisted practice, unassisted human practice, and passive observation of AI-generated exemplars on subsequent unassisted writing performance. Contribution/Results: The AI-assisted group significantly outperformed the unassisted group in post-intervention writing assessments (Cohen’s *d* = 0.40*** and 0.31***), with gains sustained for at least 24 hours; no significant difference emerged between the AI-assisted and observation-only groups. These findings indicate that learning via high-quality, immediate, and editable AI exemplars—i.e., example-based learning—is the primary mechanism underlying skill transfer. This is the first empirical study to demonstrate that AI writing tools can improve writing proficiency by optimizing pedagogical processes, thereby challenging the prevailing intuition that AI undermines skill development.

Technology Category

• Humans and AI: Learning Human Values and Preferences
• Cognitive Modeling & Cognitive Systems: Computational Creativity
• Intelligent Robots: Embodied AI

Application Category

• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

It is widely believed that outsourcing cognitive work to AI boosts immediate productivity at the expense of long-term human capital development. An opposing possibility is that AI tools can support skill development by providing just-in-time, high-quality, personalized examples. This work explores whether using an AI writing tool undermines or supports performance on later unaided writing. In Study 1, forecasters predicted that practicing writing cover letters with an AI tool would impair learning compared to practicing alone. However, in Study 2, participants randomly assigned to practice writing with AI improved more on a subsequent writing test than those assigned to practice without AI (d = 0.40***) -- despite exerting less effort, whether measured by time on task, keystrokes, or subjective ratings. In Study 3, participants who had practiced writing with AI again outperformed those who practiced without AI (d = 0.31***). Consistent with the positive impact of exposure to high-quality examples, these participants performed just as well as those who viewed -- but could not edit -- an AI-generated cover letter (d = 0.03, ns). In both Studies 2 and 3, the benefits of practicing with AI persisted in a one-day follow-up writing test. Collectively, these findings constitute an existence proof that, contrary to participants' intuition, using AI tools can improve, rather than undermine, learning.

Problem

• AI enhances skill development
• AI improves unaided writing
• High-quality examples aid learning

Innovation

• AI tools enhance skill development
• Just-in-time personalized learning examples
• Improved writing performance with AI