LLAMA: Multi-Feedback Smart Contract Fuzzing Framework with LLM-Guided Seed Generation

2025-07-16

AI Summary

Existing smart contract fuzzing tools largely neglect mutation scheduling—a critical component of effective fuzzing. Method: This paper introduces the first LLM-integrated, multi-feedback-driven fuzzing framework. It employs hierarchical prompting to generate semantically valid initial seeds; synergistically leverages runtime coverage feedback and dependency-aware feedback to jointly optimize seed selection, generation, and mutation scheduling; and incorporates evolutionary mutation, hybrid testing, and lightweight pre-fuzzing filtering to prioritize high-potential inputs. Contribution/Results: The approach establishes the first end-to-end fuzzing pipeline that unifies LLM guidance with multi-feedback optimization. Evaluated on standard benchmarks, it achieves 91% instruction coverage and 90% branch coverage, detecting 132 out of 148 known vulnerabilities—surpassing state-of-the-art tools in both coverage and vulnerability detection rate.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Planning, Routing, and Scheduling: Planning with Language Models
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Smart contracts play a pivotal role in blockchain ecosystems, and fuzzing remains an important approach to securing smart contracts. Even though mutation scheduling is a key factor influencing fuzzing effectiveness, existing fuzzers have primarily explored seed scheduling and generation, while mutation scheduling has been rarely addressed by prior work. In this work, we propose a Large Language Models (LLMs)-based Multi-feedback Smart Contract Fuzzing framework (LLAMA) that integrates LLMs, evolutionary mutation strategies, and hybrid testing techniques. Key components of the proposed LLAMA include: (i) a hierarchical prompting strategy that guides LLMs to generate semantically valid initial seeds, coupled with a lightweight pre-fuzzing phase to select high-potential inputs; (ii) a multi-feedback optimization mechanism that simultaneously improves seed generation, seed selection, and mutation scheduling by leveraging runtime coverage and dependency feedback; and (iii) an evolutionary fuzzing engine that dynamically adjusts mutation operator probabilities based on effectiveness, while incorporating symbolic execution to escape stagnation and uncover deeper vulnerabilities. Our experiments demonstrate that LLAMA outperforms state-of-the-art fuzzers in both coverage and vulnerability detection. Specifically, it achieves 91% instruction coverage and 90% branch coverage, while detecting 132 out of 148 known vulnerabilities across diverse categories. These results highlight LLAMA's effectiveness, adaptability, and practicality in real-world smart contract security testing scenarios.

Problem

• Enhancing smart contract fuzzing via LLM-guided seed generation
• Optimizing mutation scheduling for improved fuzzing effectiveness
• Integrating multi-feedback mechanisms for vulnerability detection

Innovation

• LLMs guide semantically valid seed generation
• Multi-feedback optimizes seed and mutation scheduling
• Evolutionary fuzzing adjusts mutation dynamically