Building Software by Rolling the Dice: A Qualitative Study of Vibe Coding

2025-12-26

AI Summary

This study addresses the emergent software development practice of “vibe coding”—a prompt-engineering–centric, code-generation–augmented paradigm whose cognitive and collaborative dynamics remain unexamined. Method: Drawing on grounded theory analysis of 20 authentic developer coding videos, we systematically characterize vibe coding behaviors, debug strategies, and human–AI interaction patterns. Contribution/Results: We propose a behavioral continuum for vibe coding, revealing its heavy reliance on iterative trial-and-error and regeneration—captured by the “rolling the dice” metaphor to denote paradigmatic shifts induced by generative uncertainty. We identify three distinct human–AI collaboration mental models and demonstrate that developers’ expertise and degree of AI dependence significantly shape prompting strategies and trust calibration mechanisms. This work establishes the first empirical foundation and conceptual framework for designing AI-powered programming tools, transforming computing education, and advancing theories of human–AI co-production.

Technology Category

• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior
• Humans and AI: Other Foundations of Human Computation & AI
• Natural Language Processing: Prompt Engineering / Prompting

Application Category

• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Large language models (LLMs) are reshaping software engineering by enabling "vibe coding," in which developers build software primarily through prompts rather than writing code. Although widely publicized as a productivity breakthrough, little is known about how practitioners actually define and engage in these practices. To shed light on this emerging phenomenon, we conducted a grounded theory study of 20 vibe-coding videos, including 7 live-streamed coding sessions (about 16 hours, 254 prompts) and 13 opinion videos (about 5 hours), supported by additional analysis of activity durations and prompt intents. Our findings reveal a spectrum of behaviors: some vibe coders rely almost entirely on AI without inspecting code, while others examine and adapt generated outputs. Across approaches, all must contend with the stochastic nature of generation, with debugging and refinement often described as "rolling the dice." Further, divergent mental models, shaped by vibe coders' expertise and reliance on AI, influence prompting strategies, evaluation practices, and levels of trust. These findings open new directions for research on the future of software engineering and point to practical opportunities for tool design and education.

Problem

• Investigates how developers use LLMs for vibe coding in software engineering
• Examines the spectrum of behaviors and mental models in AI-assisted coding
• Explores the stochastic nature of AI generation and its impact on debugging

Innovation

• Analyzed vibe coding through video data and grounded theory
• Identified spectrum of AI reliance and code inspection behaviors
• Explored mental models shaping prompting and trust in AI