🤖 AI Summary
In heterosexual male-dominated Free/Libre and Open Source Software (FLOSS) communities, women and LGBTQ+ developers frequently encounter sexual orientation and gender identity discrimination (SGID) discourse, severely impeding their participation. Manual moderation of vast issue-comment volumes is infeasible, necessitating automated detection. This paper introduces SGID4SE—the first SGID detection framework tailored to software engineering text. It innovatively integrates six-step domain-adapted preprocessing, comparative evaluation of ten algorithms, and six minority-class performance enhancement strategies, with fine-tuned BERT at its core and integrated class-imbalance mitigation techniques. Ten-fold cross-validation demonstrates that the model achieves 85.9% precision, 80.0% recall, and 82.9% F1-score for the SGID class, with an overall accuracy of 95.7% and Matthews Correlation Coefficient (MCC) of 80.4%, significantly outperforming baseline approaches.
📝 Abstract
In an industry dominated by straight men, many developers representing other gender identities and sexual orientations often encounter hateful or discriminatory messages. Such communications pose barriers to participation for women and LGBTQ+ persons. Due to sheer volume, manual inspection of all communications for discriminatory communication is infeasible for a large-scale Free Open-Source Software (FLOSS) community. To address this challenge, this study aims to develop an automated mechanism to identify Sexual orientation and Gender identity Discriminatory (SGID) texts from software developers' communications. On this goal, we trained and evaluated SGID4SE ( Sexual orientation and Gender Identity Discriminatory text identification for (4) Software Engineering texts) as a supervised learning-based SGID detection tool. SGID4SE incorporates six preprocessing steps and ten state-of-the-art algorithms. SGID4SE implements six different strategies to improve the performance of the minority class. We empirically evaluated each strategy and identified an optimum configuration for each algorithm. In our ten-fold cross-validation-based evaluations, a BERT-based model boosts the best performance with 85.9% precision, 80.0% recall, and 82.9% F1-Score for the SGID class. This model achieves 95.7% accuracy and 80.4% Matthews Correlation Coefficient. Our dataset and tool establish a foundation for further research in this direction.
