SoK: Where to Fuzz? Assessing Target Selection Methods in Directed Fuzzing

2024-07-01

ACM Asia Conference on Computer and Communications Security

AI Summary

Target selection in directed fuzzing has long lacked systematic investigation. Method: This paper conducts the first large-scale, quantitative evaluation of over a dozen target-selection strategies on 1,600+ real-world crash-inducing inputs; introduces lightweight code metrics—e.g., cyclomatic complexity and call depth—as principled alternatives to heuristic approaches; validates the effectiveness and generalizability of large language model (LLM)-assisted scoring for target prioritization; and treats target selection as an orthogonal optimization dimension, decoupled from instrumentation and mutation mechanisms. Contributions/Results: The proposed metrics outperform state-of-the-art heuristics, yielding a 23% average coverage improvement on OSS-Fuzz. The study integrates software metric analysis, crash-driven evaluation, sanitizer-based comparative experiments, and a cross-conference literature review, establishing both theoretical foundations and practical guidelines for directed fuzzing.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Reasoning under Uncertainty: Stochastic Optimization
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Security and Privacy: Large-scale security measurements
• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success

Abstract

A common paradigm for improving fuzzing performance is to focus on selected regions of a program rather than its entirety. While previous work has largely explored how these locations can be reached, their selection, that is, the where, has received little attention so far. In this paper, we fill this gap and present the first comprehensive analysis of target selection methods for fuzzing. To this end, we examine papers from leading security and software engineering conferences, identifying prevalent methods for choosing targets. By modeling these methods as general scoring functions, we are able to compare and measure their efficacy on a corpus of more than 1,600 crashes from the OSS-Fuzz project. Our analysis provides new insights for target selection in practice: First, we find that simple software metrics significantly outperform other methods, including common heuristics used in directed fuzzing, such as recently modified code or locations with sanitizer instrumentation. Next to this, we identify language models as a promising choice for target selection. In summary, our work offers a new perspective on directed fuzzing, emphasizing the role of target selection as an orthogonal dimension to improve performance.

Problem

• Analyzes target selection in directed fuzzing
• Compares efficacy of various selection methods
• Identifies effective metrics for target selection

Innovation

• Comprehensive analysis of target selection
• Modeling methods as scoring functions
• Using language models for selection