Engineering Students' Usage and Perceptions of GitHub Copilot in Open-Source Projects

2026-04-12

AI Summary

This study investigates behavioral differences and influencing factors in how engineering students utilize GitHub Copilot during authentic open-source contributions. Through an embedded course-based empirical design, the research systematically analyzes students’ usage patterns and subjective evaluations of Copilot’s multimodal features—including chat interaction, code generation, comment-driven suggestions, and repository-aware recommendations—by integrating survey responses with real task data. Findings indicate that the chat and code generation functionalities are the most preferred, and that gender, programming proficiency, and familiarity with AI significantly affect both usage frequency and perceived usefulness. This work provides the first empirical evidence of differential adoption behaviors among engineering students engaging with AI-powered programming assistants in real-world open-source contexts, offering a foundational understanding of human–AI collaboration in engineering education.

Technology Category

• Humans and AI: User Experience and Usability
• Natural Language Processing: Prompt Engineering / Prompting
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior

Application Category

• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

The evolution of LLM has resulted in coding-focused models that are able to produce code snippets with high accuracy. More and more AI coding assistant tools are now available, leading to greater integration of AI coding assistants into integrated development environments (IDEs). These tools introduce new possibilities for enhancing software development workflows and changing programming processes. GitHub Copilot, a popular AI coding assistant, offers features including inline code autocompletion, comment-driven code generation, repository-aware suggestions, and a chat interface for code explanation and debugging. Different users use these tools differently due to differences in their perception, prior experience, and demographics. Furthermore, differences in feature use may affect users' programming process and skills, especially for programming learners such as computer science students. While prior work has evaluated the performance of LLM-driven code generation tools, their use and usefulness for developers, especially computer science students, remain underexplored. For our investigation, we conducted an exploratory survey-based study in which participants completed a survey after completing an open-source project issue using GitHub Copilot as part of a course. We analyzed students' use of each feature and their perceived usefulness. Further, we explore and analyze significant differences in GitHub Copilot usage and students' perceptions of it based on demographic factors. Our results show that students used the GitHub Copilot chat feature and code generation feature more than other features. Gender, programming proficiency, and familiarity with AI impacted the usage of the GitHub Copilot feature for assistance in completing the open-source project contribution.

Problem

• GitHub Copilot
• AI coding assistant
• student perception
• open-source projects
• demographic factors

Innovation

• GitHub Copilot
• AI coding assistant
• student perception
• open-source contribution
• demographic factors