The Role of LLMs in Collaborative Software Design

2026-04-10

AI Summary

This study investigates the mechanisms and impacts of large language models (LLMs) in collaborative software design involving multiple participants. Through a controlled laboratory experiment, the authors observed 18 pairs of professional developers as they freely employed an LLM to design a campus bicycle parking application. The research systematically uncovers novel usage patterns—namely shared instantiation and parallel utilization of the LLM in collaborative settings—and identifies emergent phenomena such as contextual drift and early anchoring effects. Findings indicate that LLMs can enhance mutual understanding and stimulate design insights, yet excessive reliance may constrain the exploration of alternative solutions. Emphasizing human-centered design principles, this work provides empirical evidence and practical guidance for developing AI tools that effectively support collaborative software design.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Humans and AI: User Experience and Usability
• Natural Language Processing: (Large) Language Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

While much prior work examines Large Language Models (LLMs) for solo development tasks (e.g., coding), far less is known about how LLMs shape collaborative group work in software engineering. This study focuses on one such collaborative task, namely software design. It presents the results of an exploratory laboratory study of 18 pairs of software professionals who could use an LLM however they saw fit, to design a University campus bicycle parking application. Our findings reveal that introducing an LLM leads to distinct patterns of joint use: shared-instance use facilitated shared understanding, whereas parallel use across separate instances sometimes led to''context drift''. We also observe wide variation in reliance, from non-use to treating the LLM as an information source or producer. Across these modes, professionals scrutinized and reflected on LLM responses, often yielding design insights; however, early anchoring sometimes curtailed exploration. We provide implications for tools to aid designers while retaining the human-centricity important to design.

Problem

• Large Language Models
• collaborative software design
• software engineering
• human-AI collaboration

Innovation

• Large Language Models
• Collaborative Software Design
• Human-AI Collaboration
• Context Drift
• Design Exploration