How Do Software Professionals Evaluate AI-Generated Code? (Registered Report)

2026-07-10

AI Summary

This study addresses the current lack of a systematic theoretical framework explaining how software professionals evaluate AI-generated code and the underlying cognitive processes and preferences involved. Employing a constructivist grounded theory approach, the research integrates questionnaire surveys, semi-structured interviews, and laddering interviews to iteratively collect data from 20 to 50 practitioners until theoretical saturation is achieved. The work presents the first empirically grounded theoretical framework for assessing AI-generated code, elucidating the evaluation mechanisms developers employ in human-AI collaborative programming contexts. By doing so, it fills a critical gap in the literature concerning both behavioral and cognitive dimensions of code evaluation in AI-assisted software development.

Abstract

Recent advances in generative AI tools have significantly changed how software professionals write, evaluate, and interact with code. Generative AI tools such as GitHub Copilot, ChatGPT, and Claude are increasingly being integrated into everyday workflows. Despite the growing adoption of and reliance on these tools, it remains unclear as to how software professionals evaluate the code they generate. To explore this topic, we will conduct a constructivist grounded theory study that incorporates a survey, semi-structured interviews, and laddering interviews. With the initial survey data collection complete, we aim to interview 20--50 software professionals iteratively until theoretical saturation is achieved. This research aims to build a theory of how software professionals evaluate AI-generated code, grounded in their accounts of evaluative practices, perceptions, and preferences.

Problem

• AI-generated code
• software professionals
• code evaluation
• generative AI
• human-AI interaction

Innovation

• AI-generated code
• constructivist grounded theory
• code evaluation
• software professionals
• laddering interviews