BioDefect: The First Dataset for Defect Detection in Bioinformatics Software

2026-05-20

AI Summary

This study addresses the limitations of existing defect detection methods in bioinformatics, which are hindered by the absence of high-quality, domain-specific datasets. To bridge this gap, we introduce BioDefect, the first defect detection dataset tailored for bioinformatics software, constructed from real-world code repositories with full contextual preservation and rigorous mitigation of label inconsistency and data leakage. We systematically evaluate BioDefect on nine language models, including DeepSeek-R1, demonstrating substantial performance gains: all models achieve an average F1-score improvement ranging from 29.61% to 38.04% over those trained on existing general-purpose datasets. These results underscore the effectiveness and superiority of BioDefect in advancing defect detection within the bioinformatics domain.

Abstract

Software defect detection is a critical task in software engineering. However, no prior studies have specifically addressed defect detection in bioinformatics software. Given that the performance of defect detection tasks is primarily influenced by both models and datasets, our experiments controlled for model-related factors and confirmed the limitations of existing datasets in bioinformatics software. To address this issue, we introduce BioDefect, the first dataset specifically designed for defect detection in bioinformatics software, aiming to overcome the limitations of existing datasets in this context. Unlike prior datasets, BioDefect includes complete source code repositories, preserving the actual contextual information of defective code, thereby more accurately reflecting real-world defect scenarios in bioinformatics software. Additionally, BioDefect mitigates issues related to label inconsistency and data leakage, ensuring high data quality and experimental reliability. To evaluate the effectiveness of BioDefect, we conduct a systematic assessment on nine language models (LMs), including DeepSeek-R1. The results demonstrate that BioDefect significantly enhances defect detection performance for bioinformatics software. Compared to existing datasets, BioDefect achieves an average F1-score improvement of 29.61% to 38.04% across all models, highlighting its superior advantages. This study fills a critical research gap in bioinformatics software defect detection, laying a foundation for future studies in this field and offering new insights for improving bioinformatics software quality assurance.

Problem

• bioinformatics software
• defect detection
• dataset limitation
• label inconsistency
• data leakage

Innovation

• BioDefect
• defect detection
• bioinformatics software
• dataset
• code context