A Systematic Literature Review on Task Recommendation Systems for Crowdsourced Software Engineering

2024-07-13

arXiv.org

AI Summary

To address the challenge of efficiently matching practitioners with suitable tasks in crowdsourced software engineering (CSE), this study conducts a systematic literature review (SLR) following the Kitchenham & Charters guidelines, identifying and analyzing 63 primary studies. The work introduces the first taxonomy of CSE task recommendation systems, categorizing data sources into four types, recommendation tasks into five classes, and key platform characteristics; it further identifies human factors—including developer expertise, preferences, and collaborative behaviors—as central to effective recommendation. The review systematically synthesizes strengths and limitations of existing approaches. Collectively, these findings establish a structured theoretical framework for CSE task recommendation and propose actionable future research directions: integrating human-factor modeling, cross-platform feature transfer, and dynamic feedback mechanisms.

Technology Category

• Humans and AI: Crowd Sourcing and Human Computation
• Data Mining & Knowledge Management: Recommender Systems
• Search and Optimization: Metareasoning and Metaheuristics

Application Category

• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Economics and fairness of platforms and recommendation systems

Abstract

Context: Crowdsourced Software Engineering CSE offers outsourcing work to software practitioners by leveraging a global online workforce. However these software practitioners struggle to identify suitable tasks due to the variety of options available. Hence there have been a growing number of studies on introducing recommendation systems to recommend CSE tasks to software practitioners. Objective: The goal of this study is to analyze the existing CSE task recommendation systems, investigating their extracted data, recommendation methods, key advantages and limitations, recommended task types, the use of human factors in recommendations, popular platforms, and features used to make recommendations. Method: This SLR was conducted according to the Kitchenham and Charters guidelines. We used both manual and automatic search strategies without putting any time limitation for searching the relevant papers. Results: We selected 63 primary studies for data extraction, analysis, and synthesis based on our predefined inclusion and exclusion criteria. From the results of the data analysis, we classified the extracted data into 4 categories based on the data extraction source, categorized the proposed recommendation systems to fit into a taxonomy, and identified the key advantages and limitations of these systems. Our results revealed that human factors play a major role in CSE task recommendation. Further we identified five popular task types recommended, popular platforms, and their features used in task recommendation. We also provided recommendations for future research directions. Conclusion: This SLR provides insights into current trends gaps and future research directions in CSE task recommendation systems.

Problem

• Analyzes CSE task recommendation systems for software practitioners.
• Investigates data, methods, advantages, limitations, and human factors.
• Identifies popular task types, platforms, and features for recommendations.

Innovation

• Systematic review of CSE task recommendation systems
• Classification of data into four source categories
• Analysis of human factors in task recommendations