Automated Vulnerability Injection in Solidity Smart Contracts: A Mutation-Based Approach for Benchmark Development

2025-04-22

AI Summary

Existing smart contract security benchmarks are outdated and exhibit narrow vulnerability coverage. Method: This paper proposes an automated vulnerability injection framework based on pattern-aware mutation operators, designed to inject six representative vulnerability classes into real-world Solidity contracts at scale. The approach integrates Solidity syntactic and semantic constraints, devising vulnerability-aware mutation operators and an injection engine; it further presents the first systematic feasibility and limitation study of mutation-based seeding for smart contracts. Contribution/Results: We construct a large-scale, diverse benchmark dataset and evaluate it using Slither. Our method generates hundreds of vulnerable contracts with realistic flaws, exposing significant detection gaps in mainstream static analyzers—Slither exhibits an average false-negative rate of 38.7% on mutated vulnerabilities. This work establishes a scalable, semantically grounded benchmarking framework for rigorously assessing the robustness of smart contract static analysis tools.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories
• Natural Language Processing: Safety and Robustness

Application Category

• Security and Privacy: Cryptocurrency and smart contracts
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Web Mining and Content Analysis: Web data provenance, reliability, and authenticity

Abstract

The security of smart contracts is critical in blockchain systems, where even minor vulnerabilities can lead to substantial financial losses. Researchers proposed several vulnerability detection tools evaluated using existing benchmarks. However, most benchmarks are outdated and focus on a narrow set of vulnerabilities. This work evaluates whether mutation seeding can effectively inject vulnerabilities into Solidity-based smart contracts and whether state-of-the-art static analysis tools can detect the injected flaws. We aim to automatically inject vulnerabilities into smart contracts to generate large and wide benchmarks. We propose MuSe, a tool to generate vulnerable smart contracts by leveraging pattern-based mutation operators to inject six vulnerability types into real-world smart contracts. We analyzed these vulnerable smart contracts using Slither, a static analysis tool, to determine its capacity to identify them and assess their validity. The results show that each vulnerability has a different injection rate. Not all smart contracts can exhibit some vulnerabilities because they lack the prerequisites for injection. Furthermore, static analysis tools fail to detect all vulnerabilities injected using pattern-based mutations, underscoring the need for enhancements in static analyzers and demonstrating that benchmarks generated by mutation seeding tools can improve the evaluation of detection tools.

Problem

• Evaluates mutation seeding for injecting vulnerabilities in Solidity contracts
• Assesses static analysis tools' ability to detect injected vulnerabilities
• Aims to generate large benchmarks for vulnerability detection tools

Innovation

• Mutation-based vulnerability injection in Solidity
• Pattern-based mutation operators for vulnerabilities
• Automated benchmark generation for static analyzers