Exploring Creativity in Human-Human-LLM Collaborative Software Design

2026-04-27

AI Summary

This study investigates the impact of large language models (LLMs) on human creativity in collaborative software design. Through a controlled laboratory simulation, 18 pairs of experienced software engineers completed a 90-minute design task with optional access to an LLM interface. Employing a custom interaction platform, behavioral observation, and qualitative content analysis, the researchers systematically coded and evaluated creative interactions. The findings reveal, for the first time, the dual role of LLMs in authentic collaborative settings: they can either stimulate novel ideas or extend human conceptualization, yet may also suppress creative emergence through overly complex suggestions. Crucially, human expertise, empathy, and analogical reasoning remain central to creative output. Thirteen teams produced creatively distinctive design documents, and the effectiveness of LLM assistance was found to be highly contingent upon proactive human guidance.

Technology Category

• Cognitive Modeling & Cognitive Systems: Computational Creativity
• Machine Learning: Large Multimodal Models (LMMs)
• Humans and AI: Game Design — Virtual Humans, NPCs and Autonomous Characters

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

While the use of Large Language Models (LLMs) in programming has been extensively studied, there is limited understanding of how LLMs support collaborative work where creativity plays a central role. Software design, as a collaborative and creative activity, provides a valuable context for exploring the influence of LLMs on creativity. This study investigates how and where creativity naturally emerges when software designers collaborate with an LLM during a design task. In a laboratory setting simulating a workplace environment, 18 pairs of software professionals with design experience were asked to complete a design task. Each pair had 90 minutes to produce a software design based on a set of requirements, with optional access to a custom LLM interface. Pairs were not primed to be creative. We find that creativity was present in all pairs in design processes, with 13 producing design documents containing creativity. We primarily attribute creativity to the human designers, driven by traits such as prior experience, empathy, and the use of analogies. The LLM contributed by producing novel ideas and elaborating human ideas. However, in some cases, the LLM appeared to hinder creativity by suggesting complex solutions or adding to unproductive digressions. LLMs can support creativity in collaborative software design, but human insights remain central. To effectively augment human creativity, designers must be intentional in their engagement with LLMs.

Problem

• creativity
• Large Language Models
• collaborative software design
• human-AI collaboration
• software design

Innovation

• human-AI collaboration
• creativity
• Large Language Models
• software design
• empirical study