Unlimited Practice Opportunities: Automated Generation of Comprehensive, Personalized Programming Tasks

2025-03-12

AI Summary

To address the lack of personalized programming practice in computer science education, this work proposes an end-to-end generative AI (GenAI)-based framework for adaptive programming task generation, implemented as a new module in the Tutor Kai system. The framework enables students to specify target concepts and topics, then automatically generates complete, pedagogically sound tasks—including problem descriptions, starter code skeletons, unit tests, and reference solutions—while jointly ensuring functional correctness, solvability, and multi-concept coherence. Evaluation involved expert assessment (N=40 tasks) and a student study (N=62). Results show 89.5% of generated tasks are functionally complete and 92.5% are solvable; students reported high satisfaction and significantly improved perceived learning gains. This work contributes a scalable, empirically validated methodology for GenAI-driven adaptive programming instruction.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Humans and AI: Game Design — Procedural Content Generation & Storytelling
• Cognitive Modeling & Cognitive Systems: Computational Creativity

Application Category

• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience

Abstract

Generative artificial intelligence (GenAI) offers new possibilities for generating personalized programming exercises, addressing the need for individual practice. However, the task quality along with the student perspective on such generated tasks remains largely unexplored. Therefore, this paper introduces and evaluates a new feature of the so-called Tutor Kai for generating comprehensive programming tasks, including problem descriptions, code skeletons, unit tests, and model solutions. The presented system allows students to freely choose programming concepts and contextual themes for their tasks. To evaluate the system, we conducted a two-phase mixed-methods study comprising (1) an expert rating of 200 automatically generated programming tasks w.r.t. task quality, and (2) a study with 26 computer science students who solved and rated the personalized programming tasks. Results show that experts classified 89.5% of the generated tasks as functional and 92.5% as solvable. However, the system's rate for implementing all requested programming concepts decreased from 94% for single-concept tasks to 40% for tasks addressing three concepts. The student evaluation further revealed high satisfaction with the personalization. Students also reported perceived benefits for learning. The results imply that the new feature has the potential to offer students individual tasks aligned with their context and need for exercise. Tool developers, educators, and, above all, students can benefit from these insights and the system itself.

Problem

• Generates personalized programming exercises using GenAI
• Evaluates task quality and student satisfaction with generated tasks
• Assesses system's ability to implement multiple programming concepts

Innovation

• Generative AI for personalized programming exercises
• Tutor Kai generates tasks with descriptions, code, tests
• Two-phase study evaluates task quality and student satisfaction