Large-Scale Empirical Analysis of Continuous Fuzzing: Insights from 1 Million Fuzzing Sessions

2025-10-18

AI Summary

The practical efficacy of continuous fuzzing in vulnerability detection remains poorly understood. Method: We conduct a cross-project quantitative analysis based on 1.12 million real-world fuzzing sessions from OSS-Fuzz, integrating coverage reports, fuzzing logs, and vulnerability bug reports. Contribution/Results: Our large-scale empirical study reveals that vulnerability discovery is highly concentrated in the early phase of continuous fuzzing—over 60% of vulnerabilities are detected within the first seven days. Moreover, we identify a strong dynamic correlation between sustained coverage growth and novel vulnerability discovery: iterative expansion of coverage boundaries directly drives subsequent vulnerability identification. These findings empirically validate the effectiveness of continuous fuzzing and provide data-driven guidance for optimizing operational strategies—including resource allocation and termination criteria—thereby filling a critical gap in evidence-based evaluation of fuzzing practices.

Technology Category

• Search and Optimization: Mixed Discrete/Continuous Search
• Reasoning under Uncertainty: Stochastic Optimization
• Data Mining & Knowledge Management: Anomaly/Outlier Detection

Application Category

• Security and Privacy: Large-scale security measurements
• Web Mining and Content Analysis: Web data provenance, reliability, and authenticity
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Software vulnerabilities are constantly being reported and exploited in software products, causing significant impacts on society. In recent years, the main approach to vulnerability detection, fuzzing, has been integrated into the continuous integration process to run in short and frequent cycles. This continuous fuzzing allows for fast identification and remediation of vulnerabilities during the development process. Despite adoption by thousands of projects, however, it is unclear how continuous fuzzing contributes to vulnerability detection. This study aims to elucidate the role of continuous fuzzing in vulnerability detection. Specifically, we investigate the coverage and the total number of fuzzing sessions when fuzzing bugs are discovered. We collect issue reports, coverage reports, and fuzzing logs from OSS-Fuzz, an online service provided by Google that performs fuzzing during continuous integration. Through an empirical study of a total of approximately 1.12 million fuzzing sessions from 878 projects participating in OSS-Fuzz, we reveal that (i) a substantial number of fuzzing bugs exist prior to the integration of continuous fuzzing, leading to a high detection rate in the early stages; (ii) code coverage continues to increase as continuous fuzzing progresses; and (iii) changes in coverage contribute to the detection of fuzzing bugs. This study provides empirical insights into how continuous fuzzing contributes to fuzzing bug detection, offering practical implications for future strategies and tool development in continuous fuzzing.

Problem

• Evaluating continuous fuzzing's role in detecting software vulnerabilities
• Analyzing code coverage and session frequency during bug discovery
• Investigating empirical data from 1.12 million fuzzing sessions

Innovation

• Analyzing 1.12 million fuzzing sessions from OSS-Fuzz
• Studying code coverage growth and bug detection patterns
• Providing empirical insights for continuous fuzzing strategies