Secret Breach Prevention in Software Issue Reports
This study addresses the previously unexplored problem of inadvertent leakage of sensitive information—such as API keys and authentication tokens—in GitHub issue reports. To overcome the lack of benchmark datasets and effective detection methods for this scenario, we introduce the first large-scale, manually annotated benchmark containing 5,800 real-world secrets. We propose a lightweight hybrid detection paradigm that synergistically combines regex-based extraction with large language model (LLM)-driven contextual classification. Our method integrates entropy analysis, RoBERTa/CodeBERT feature encoding, and fine-tuned Qwen/LLaMA models, augmented by GPT-4o few-shot learning to enhance generalization. Evaluated on our benchmark, the approach achieves an F1 score of 94.49%; it further attains 81.6% F1 across 178 real-world repositories, significantly outperforming conventional entropy- and keyword-based baselines. This work establishes the first dedicated framework for detecting secret leakage in issue reports, demonstrating the superiority of open-source LLM fine-tuning for this task.