The Emerging Use of GenAI for UX Research in Software Development: Challenges and Opportunities

2025-12-17

AI Summary

This study addresses a core tension in applying generative AI to qualitative UX research in software development: the opacity of AI systems conflicts with qualitative research’s inherent demands for interpretability and collaborative sensemaking, while role-based differences between UX researchers and product managers exacerbate trust deficits and cognitive misalignment. Using semi-structured interviews, human-centered AI process modeling, and cross-role workflow mapping, we identify three critical challenges: human-AI collaboration friction, ambiguous accountability, and role misalignment. We propose a novel “bottom-up” explainable AI (XAI) analytical framework, coupled with an agile-oriented UX-AI co-interaction model and a responsibility-integrated design guideline. Empirical evaluation demonstrates that this paradigm significantly enhances the credibility of AI-generated insights and improves team-wide alignment in interpretation and decision-making, effectively bridging the trust gap between stakeholders.

Technology Category

• Humans and AI: User Experience and Usability
• Philosophy and Ethics of AI: Accountability, Interpretability & Explainability
• Knowledge Representation and Reasoning: Qualitative Reasoning

Application Category

• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• User Modeling, Personalization and Recommendation: Accountability, Transparency, and Ethics for personalization

Abstract

The growing adoption of generative AI (GenAI) is reshaping how user experience (UX) research teams conduct qualitative research in software development, creating opportunities to streamline the production of qualitative insights. This paper presents findings from two user studies examining how current practices are challenged by GenAI and offering design implications for future AI assistance. Semi-structured interviews with 21 UX researchers, product managers, and designers reveal challenges of aligning AI capabilities with the interpretive, collaborative nature of qualitative research and tensions between roles. UX researchers expressed limited trust in AI-generated results, while product managers often overestimated AI capabilities, amplifying organizational pressures to accelerate research within agile workflows. In a second study, we validated an AI analysis approach more closely aligned with human analysis processes to address trust issues bottoms-up. We outline interaction patterns and design guidelines for responsibly integrating AI into software development cycles.

Problem

• Addresses challenges of integrating GenAI into UX research workflows
• Explores trust issues between AI-generated results and human interpretation
• Proposes design guidelines for AI alignment with qualitative research processes

Innovation

• Validated AI analysis approach aligning with human processes
• Developed interaction patterns for responsible AI integration
• Created design guidelines addressing trust and organizational tensions