GitReq: A Gold Standard Dataset for Software Quality Requirements
This work addresses the scarcity of fine-grained, expert-annotated non-functional requirement (NFR) samples in existing public GitHub datasets by introducing GitReq—the first large-scale GitHub-based quality requirements dataset aligned with the ISO/IEC 25010 standard. GitReq comprises 6,302 expert-validated requirements extracted from 4,080 repositories, spanning all eight quality characteristics defined in the standard. The construction methodology employs category-specific tri-signal mining, a preprocessing step to separate functional from non-functional requirements, and a rigorous manual annotation protocol, achieving a Fleiss’ Kappa inter-annotator agreement of 0.72. Experimental evaluation demonstrates that GPT-5.2 attains a macro-averaged F1 score of 0.641 under zero-shot settings, confirming both the dataset’s validity and its inherent challenge for current language models.