How Developers Discuss Generative AI: A Longitudinal Study of the Visual Studio Code Community

2026-09-03

AI Summary

研究通过分析Visual Studio Code社区的讨论，使用关键词检索和语义过滤结合的方法，探讨了开发者如何讨论生成式AI工具及其在软件开发中的实际应用问题。

Abstract

Generative AI tools such as GitHub Copilot, ChatGPT, and coding agents have rapidly become part of everyday software development, yet little is known about how mainstream open source communities discuss them in practice. This paper presents a longitudinal analysis of generative-AI-related discussions in the Visual Studio Code (VS Code) GitHub repository, using 43,806 candidate issues created between January 2021 and June 2026. To improve corpus quality, we combined keyword retrieval with semantic relevance filtering, yielding a filtered corpus of 25,227 AI-related issues. We applied BERTopic to the retrieved corpus to identify discussion topics, using the filtered corpus for theme validation and a robustness re-clustering, and analyzed their evolution over time using monthly prevalence and Mann-Kendall trend tests. The results show that developer discussions are dominated by practical concerns regarding the operation of AI-assisted development environments, including agent management, configuration, reliability, authentication, and billing, whereas risks frequently emphasized in survey-based studies, such as hallucination and licensing, rarely surface in this venue. This suggests that discussions of generative AI in the VS Code issue tracker primarily focus on operational aspects of AI-assisted software development. Furthermore, discussions evolved from AI-assisted code completion toward conversational and agent-based development, reflecting the increasing integration of generative AI into software development workflows. These findings suggest that GitHub Issues provide a practical, workflow-oriented perspective on generative AI that complements survey-based studies of developer perceptions.

Problem

• generative AI
• developer discussions
• open source communities
• Visual Studio Code
• GitHub Issues

Innovation

• keyword retrieval and semantic relevance filtering
• BERTopic for topic identification
• Mann-Kendall trend tests
• operational aspects of AI-assisted development
• evolution of discussions