Say What? Examining Text and Voice Input Modalities for Prompt-Based Programming in Computing Education

2026-07-07

AI Summary

This study presents the first systematic comparison of text and speech input modalities in large language model–based prompt programming instruction. By integrating platform log analysis with survey data, it examines novice learners’ behaviors, task success rates, and modality preferences. The findings reveal that unedited voice prompts yield significantly lower initial success rates than typed text; however, this gap disappears after users edit their prompts. Although most students express a preference for text input, a subset demonstrates adaptive switching between modalities, suggesting potential for complementary multimodal interaction. The work uncovers discrepancies between perceived usability and actual usage patterns, offering empirical insights to inform the design of multimodal programming education tools.

Abstract

Large language models (LLMs) are increasingly integrated into computing education, yet nearly all prior research has focused on text-based interactions. As voice-enabled interfaces become more capable and more common, there is growing interest in understanding how voice input might shape students' use of LLM-powered tools. In this exploratory study, we investigated how introductory programming students interact with Prompt Problems, which are programming tasks that require crafting natural-language prompts to generate correct code. Students (N = 919) solved a series of Prompt Problems with the freedom to select or switch between text and voice input modalities. We collected their prompt submissions as well as post-activity survey responses, then analysed differences in prompt accuracy, persistence, and perspectives by modality. For two of the three problems, we found that students who typed their prompts using text were more likely to have those prompts succeed on the first attempt than students who submitted unedited voice prompts. There was no difference in success rate if students edited their transcribed voice prompts before submission. Across the problems, we found evidence that students who tried voice prompting varied in their usage of modality - perhaps indicating a complementary, or non-preferential approach. However, most students only tried and reported preferring text. Our qualitative analysis revealed how students' perceived the roles of voice and text input in shaping their problem-solving process, as well as the reported drawbacks and advantages of each modality. We discuss implications for future multimodal tools and instructional design in computing education.

Problem

• voice input
• text input
• prompt-based programming
• computing education
• large language models

Innovation

• voice input
• text input
• prompt-based programming
• multimodal interaction
• computing education