All You Need Is A Fuzzing Brain: An LLM-Powered System for Automated Vulnerability Detection and Patching

2025-09-08

AI Summary

Addressing the challenge of automated detection and repair of unknown zero-day vulnerabilities in real-world open-source projects—particularly in C and Java—this paper introduces the first LLM-driven end-to-end security analysis system. The system integrates large language models (LLMs) with program analysis, symbolic execution, and feedback-guided fuzzing to enable closed-loop reasoning for vulnerability discovery, root-cause localization, and patch generation. It innovatively proposes an LLM-augmented fuzzing framework and releases the first open-source benchmark suite derived from DARPA’s AIxCC competition data. In AIxCC evaluations, the system identified 28 vulnerabilities—including 6 zero-days—and successfully generated and validated 14 functional patches. All source code, datasets, and benchmark infrastructure are publicly released, establishing a reproducible evaluation baseline and a novel technical paradigm for AI-augmented software security research.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Large Vision Models
• Natural Language Processing: Safety and Robustness

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Security and Privacy: Large-scale security measurements

Abstract

Our team, All You Need Is A Fuzzing Brain, was one of seven finalists in DARPA's Artificial Intelligence Cyber Challenge (AIxCC), placing fourth in the final round. During the competition, we developed a Cyber Reasoning System (CRS) that autonomously discovered 28 security vulnerabilities - including six previously unknown zero-days - in real-world open-source C and Java projects, and successfully patched 14 of them. The complete CRS is open source at https://github.com/o2lab/afc-crs-all-you-need-is-a-fuzzing-brain. This paper provides a detailed technical description of our CRS, with an emphasis on its LLM-powered components and strategies. Building on AIxCC, we further introduce a public leaderboard for benchmarking state-of-the-art LLMs on vulnerability detection and patching tasks, derived from the AIxCC dataset. The leaderboard is available at https://o2lab.github.io/FuzzingBrain-Leaderboard/.

Problem

• Automated detection of security vulnerabilities in software
• Automated patching of discovered vulnerabilities in code
• Benchmarking LLMs on vulnerability detection and patching tasks

Innovation

• LLM-powered automated vulnerability detection system
• Autonomous patching of security vulnerabilities
• Public leaderboard for LLM vulnerability benchmarking