Human in the Loop for Fuzz Testing: Literature Review and the Road Ahead

2026-03-12

AI Summary

Existing automated fuzzing approaches struggle to uncover deep, complex vulnerabilities, primarily due to the absence of effective human–machine collaboration mechanisms. This work presents the first systematic research framework dedicated to human-in-the-loop fuzzing, explicitly defining three key roles for human experts: visual monitoring, real-time intervention, and collaboration with large language models (LLMs). By integrating visualization techniques, real-time feedback loops, LLM capabilities, and expert meta-knowledge, the study establishes a novel paradigm of interactive intelligent fuzzing. Beyond synthesizing the current state of the field, the paper articulates a forward-looking research agenda, laying both theoretical and methodological foundations for next-generation fuzzing systems that synergistically combine expert insight with AI-driven automation.

Technology Category

• Humans and AI: Human-in-the-loop Machine Learning
• Intelligent Robots: Human-Robot Interaction
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling

Abstract

Fuzz testing is one of the most effective techniques for detecting bugs and vulnerabilities in software. However, as the basis of fuzz testing, automated heuristics often fail to uncover deep or complex vulnerabilities. As a result, the performance of fuzz testing remains limited. One promising way to address this limitation is to integrate human expert guidance into the paradigm of fuzz testing. Even though some works have been proposed in this direction, there is still a lack of a systematic research roadmap for combining Human-in-the-Loop (HITL) and fuzz testing, hindering the potential for further enhancing fuzzing effectiveness. To bridge this gap, this paper outlines a forward-looking research roadmap for HITL for fuzz testing. Specifically, we highlight the promise of visualization techniques for interpretable fuzzing processes, as well as on-the-fly interventions that enable experts to guide fuzzing toward hard-to-reach program behaviors. Moreover, the rise of Large Language Models (LLMs) introduces new opportunities and challenges, raising questions about how humans can efficiently provide actionable knowledge, how expert meta-knowledge can be leveraged, and what roles humans should play in the intelligent fuzzing loop with LLMs. To address these questions, we survey existing work on HITL fuzz testing and propose a research agenda emphasizing future opportunities in (1) human monitoring, (2) human steering, and (3) human-LLM collaboration. We call for a paradigm shift toward interactive, human-guided fuzzing systems that integrate expert insight with AI-powered automation in the next-generation fuzzing ecosystem.

Problem

• fuzz testing
• Human-in-the-Loop
• software vulnerabilities
• Large Language Models
• expert guidance

Innovation

• Human-in-the-Loop
• Fuzz Testing
• Large Language Models
• Interactive Debugging
• Program Analysis