Developers' Experience with Generative AI Beyond Productivity Assessment -- Insights from an Empirical Mixed-Methods Field Study

2026-07-02

AI Summary

This study addresses a critical gap in existing research by shifting focus from the impact of generative AI on development efficiency and code quality to developers’ lived interaction experiences and subjective well-being in real-world settings. Employing a mixed-methods approach—integrating controlled experiments, naturalistic observation, log analysis, and in-depth interviews—the work systematically investigates how professional developers interact with tools such as GitHub Copilot. It proposes empirically grounded heuristics for selecting AI interaction modes based on task characteristics, revealing that concurrently using code suggestions and chat prompts can diminish effectiveness. The findings identify cognitive load and output quality as pivotal factors shaping developer experience, report high overall satisfaction—particularly for repetitive tasks—and demonstrate that interaction strategy efficacy varies by task type, while participation in the study heightened developers’ intentional use of AI tools.

Abstract

With the growing adoption of AI-powered coding assistants, organizations and developers are increasingly seeking to optimize their interaction with these tools. Prior research has largely focused on output quality and productivity gains, with limited attention paid to developers' well-being and interaction experiences. This paper presents a developer-centered empirical mixed-methods study to investigate how professional developers engage with Generative AI (GenAI) in their natural work environment. Controlled data collection sessions are combined with natural work periods. Results show that developers are generally satisfied with GenAI, particularly for monotonous, repetitive, and structured tasks, and report perceived efficiency and productivity gains. Copilot interaction type preferences differ by task type and complexity: While both in-code suggestions and chat-based prompting independently improve task efficiency and reduce perceived workload, combining these interaction types within a single task diminishes benefits. We propose a rule-of-thumb for selecting an interaction type based on task characteristics. During development-heavy tasks, results indicate that perceived cognitive load arises from AI interaction, while perceived productivity depends on AI output quality. Participation in this study positively influenced developers' awareness and intentional use of GenAI tools. These findings demonstrate the value of real-world, mixed-methods study designs to understand GenAI tools and developers' experiences with them.

Problem

• Generative AI
• developer experience
• cognitive load
• AI interaction
• well-being

Innovation

• Generative AI
• developer experience
• mixed-methods study
• cognitive load
• interaction modalities