An LLM's Attempts to Adapt to Diverse Software Engineers' Problem-Solving Styles: More Inclusive&Equitable?

2025-03-14

AI Summary

Existing LLM-based code explanation methods fail to accommodate diverse problem-solving styles among software engineers, compromising inclusivity and fairness. Method: This paper pioneers the adaptation of the inclusive design framework GenderMag—originally developed for gender-inclusive software evaluation—to LLM code explanation, operationalizing its five cognitive style dimensions into a prompt engineering strategy for style-adaptive explanation generation. We conduct a mixed-methods user study (N=32) to assess how style-matched versus mismatched explanations affect comprehension performance and perceived fairness. Contribution/Results: Style-matched explanations significantly improve both code understanding and perceived fairness overall; however, certain mismatched explanations are also widely accepted, revealing the critical insight that “style matching does not universally guarantee benefit.” This work provides the first empirical foundation and a scalable methodological pathway for designing fair, inclusive AI–human interactions in programming contexts.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Ethics, Bias, and Fairness
• Humans and AI: Explainable AI (XAI) for Human Understanding

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Software engineers use code-fluent large language models (LLMs) to help explain unfamiliar code, yet LLM explanations are not adapted to engineers' diverse problem-solving needs. We prompted an LLM to adapt to five problem-solving style types from an inclusive design method, the Gender Inclusiveness Magnifier (GenderMag). We ran a user study with software engineers to examine the impact of explanation adaptations on software engineers' perceptions, both for explanations which matched and mismatched engineers' problem-solving styles. We found that explanations were more frequently beneficial when they matched problem-solving style, but not every matching adaptation was equally beneficial; in some instances, diverse engineers found as much (or more) benefit from mismatched adaptations. Through an equity and inclusivity lens, our work highlights the benefits of having an LLM adapt its explanations to match engineers' diverse problem-solving style values, the potential harms when matched adaptations were not perceived well by engineers, and a comparison of how matching and mismatching LLM adaptations impacted diverse engineers.

Problem

• LLM explanations lack adaptation to diverse problem-solving styles.
• Study examines impact of style-matched vs. mismatched LLM explanations.
• Matching adaptations benefit engineers, but mismatched ones also show value.

Innovation

• LLM adapts to diverse problem-solving styles
• GenderMag method used for inclusive design
• User study compares matched and mismatched adaptations