Exploring the Value of Diverse LLM Explanations in Introductory Programming

2026-06-27

AI Summary

This study addresses the challenge that explanations generated by large language models (LLMs) for programming tasks still fall short of expert instructor quality. It introduces, for the first time in programming education, the notion of diversity from computational creativity by leveraging LLMs to provide novice learners with varied explanations emphasizing distinct dimensions—such as functionality, underlying concepts, and learning objectives. Using a mixed-methods approach combining a randomized controlled trial, quantitative assessments, and qualitative feedback, the findings demonstrate that diverse explanations significantly enhance students’ understanding of programming exercises compared to a single generic explanation, yielding an average 7.7% improvement in accuracy on open-ended questions without increasing cognitive load. This work thus validates the unique pedagogical value of multi-perspective explanations in programming instruction.

Abstract

Large Language Models (LLMs) have shown the potential to generate code explanations that surpass those of peers in quality, offering promising opportunities for computer science education. While these explanations may not yet match the depth and clarity of instructor-provided explanations, research in computational creativity highlights that the quantity and diversity of ideas can often outweigh a singular focus on quality. Inspired by this, we explore whether combining multiple diverse explanations, each emphasizing distinct aspects (e.g., function, concept, goal), can enhance students' understanding of programming exercises compared to generic explanations that do not emphasize distinct conceptual aspects. In our study 971 first-year computing students were randomly assigned either diverse or generic LLM-generated explanations for two programming exercises. Students completed multiple-choice and open-ended questions for each exercise, followed by Likert-scale questions and open-ended reflections. Our findings outline patterns in student performance and perceived cognitive load across the two explanation conditions. These findings highlight how variation in explanation emphasis may relate to learner engagement and understanding. Across participants, open-ended response accuracy was consistently about 7.7% higher when students received diverse explanations, with no difference in perceived cognitive load.

Problem

• LLM explanations
• programming education
• explanation diversity
• student understanding
• introductory programming

Innovation

• diverse explanations
• large language models
• programming education
• explanation diversity
• cognitive load