Detecting UX smells in Visual Studio Code using LLMs

2026-02-25

AI Summary

This study addresses the challenge of systematically identifying and analyzing user experience (UX) issues in Visual Studio Code, which developers frequently encounter but lack structured methodologies to diagnose. To bridge this gap, the work proposes the first application of large language models (LLMs) to automatically mine UX defects from GitHub user reports. By integrating expert-validated categorization criteria, the authors construct a reusable UX defect classification framework. Their analysis reveals that UX problems predominantly fall into four dimensions: informativeness, clarity, intuitiveness, and efficiency. These findings effectively uncover the most salient pain points in developer experience, offering a data-driven foundation for targeted improvements in integrated development environment (IDE) UX design.

Technology Category

• Humans and AI: User Experience and Usability
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language

Application Category

• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Integrated Development Environments shape developers' daily experience, yet the empirical study of their usability and user experience (UX) remains limited. This work presents an LLM-assisted approach to detecting UX smells in Visual Studio Code by mining and classifying user-reported issues from the GitHub repository. Using a validated taxonomy and expert review, we identified recurring UX problems that affect the developer experience. Our results show that the majority of UX smells are concentrated in informativeness, clarity, intuitiveness, and efficiency, qualities that developers value most.

Problem

• UX smells
• developer experience
• Visual Studio Code
• usability
• user experience

Innovation

• LLM-assisted UX analysis
• UX smells detection
• developer experience
• Visual Studio Code
• GitHub issue mining