Exploring Student-AI Interactions in Vibe Coding

2025-07-30

AI Summary

This study investigates how novice and advanced software engineering students differ in their AI-augmented programming behaviors on the Replit platform (“vibe coding”). Method: Using concurrent think-aloud protocols paired with screen recording, we empirically observed both groups building web applications; thematic analysis was applied to identify recurring interaction patterns. Contribution/Results: (1) Both groups predominantly engaged in test–debug cycles, rarely proactively reviewing generated code. (2) Advanced students systematically embedded functional requirements and code context into prompts, significantly improving AI output relevance and usability. (3) Programming experience shifts AI collaboration from tool invocation toward contextualized co-construction. This work is the first to characterize an “experience gradient” in vibe coding—where expertise drives a qualitative evolution in AI interaction paradigms—providing empirical grounding for designing intelligent programming environments and fostering computational thinking.

Technology Category

• Humans and AI: User Experience and Usability
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Cognitive Modeling & Cognitive Systems: Computational Creativity

Application Category

• Economics, Online Markets and Human Computation: Research challenges in human and human-AI computation
• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Background and Context. Chat-based and inline-coding-based GenAI has already had substantial impact on the CS Education community. The recent introduction of ``vibe coding'' may further transform how students program, as it introduces a new way for students to create software projects with minimal oversight. Objectives. The purpose of this study is to understand how students in introductory programming and advanced software engineering classes interact with a vibe coding platform (Replit) when creating software and how the interactions differ by programming background. Methods. Interview participants were asked to think-aloud while building a web application using Replit. Thematic analysis was then used to analyze the video recordings with an emphasis on the interactions between the student and Replit. Findings. For both groups, the majority of student interactions with Replit were to test or debug the prototype and only rarely did students visit code. Prompts by advanced software engineering students were much more likely to include relevant app feature and codebase contexts than those by introductory programming students.

Problem

• Study student-AI interactions in vibe coding platforms
• Compare interactions by programming background level
• Analyze Replit usage patterns during web app development

Innovation

• Using vibe coding platform Replit
• Think-aloud interviews for interaction analysis
• Thematic analysis of student-AI interactions