HarnessAgent: Scaling Automatic Fuzzing Harness Construction with Tool-Augmented LLM Pipelines

2025-12-03

AI Summary

Automatically generating fuzzing harnesses for internal functions in large-scale C/C++ projects remains challenging due to incomplete static context, frequent compilation failures, and hallucinated code from LLMs. Method: This paper proposes a tool-augmented LLM agent framework. Its core innovations include: (1) a rule-based compilation error mitigation strategy; (2) a hybrid tool pool enabling precise symbolic retrieval; and (3) a multi-stage verification mechanism to detect logically invalid definitions. Results: Evaluated on 243 OSS-Fuzz targets, the approach achieves 87% (C) and 81% (C++) three-shot success rates—improving over baselines by 20 percentage points. 75% of generated harnesses increase code coverage, and source-code retrieval achieves >90% response rate. The framework significantly enhances adaptability and robustness for complex open-source projects.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Large language model (LLM)-based techniques have achieved notable progress in generating harnesses for program fuzzing. However, applying them to arbitrary functions (especially internal functions) extit{at scale} remains challenging due to the requirement of sophisticated contextual information, such as specification, dependencies, and usage examples. State-of-the-art methods heavily rely on static or incomplete context provisioning, causing failure of generating functional harnesses. Furthermore, LLMs tend to exploit harness validation metrics, producing plausible yet logically useless code. % Therefore, harness generation across large and diverse projects continues to face challenges in reliable compilation, robust code retrieval, and comprehensive validation. To address these challenges, we present HarnessAgent, a tool-augmented agentic framework that achieves fully automated, scalable harness construction over hundreds of OSS-Fuzz targets. HarnessAgent introduces three key innovations: 1) a rule-based strategy to identify and minimize various compilation errors; 2) a hybrid tool pool for precise and robust symbol source code retrieval; and 3) an enhanced harness validation pipeline that detects fake definitions. We evaluate HarnessAgent on 243 target functions from OSS-Fuzz projects (65 C projects and 178 C++ projects). It improves the three-shot success rate by approximately 20% compared to state-of-the-art techniques, reaching 87% for C and 81% for C++. Our one-hour fuzzing results show that more than 75% of the harnesses generated by HarnessAgent increase the target function coverage, surpassing the baselines by over 10%. In addition, the hybrid tool-pool system of HarnessAgent achieves a response rate of over 90% for source code retrieval, outperforming Fuzz Introspector by more than 30%.

Problem

• Automates fuzzing harness construction for arbitrary functions at scale
• Addresses unreliable compilation and incomplete context in harness generation
• Detects and prevents generation of plausible but useless harness code

Innovation

• Rule-based strategy to minimize compilation errors
• Hybrid tool pool for robust source code retrieval
• Enhanced validation pipeline to detect fake definitions