Validating Solidity Code Defects using Symbolic and Concrete Execution powered by Large Language Models

2025-09-16

AI Summary

Static analysis tools and large language models (LLMs) suffer from high false-positive rates in Solidity vulnerability detection, hindering reliable defect validation. To address this, we propose a multi-stage verification pipeline integrating Slither’s custom detectors, LLM-based reasoning, Kontrol’s symbolic execution, and Forge’s concrete execution—uniquely unifying heuristic analysis with formal verification through closed-loop coordination between symbolic and concrete execution. Our approach significantly reduces false positives, particularly for elusive vulnerabilities such as reentrancy, complex fallback logic, and fine-grained access control violations. Evaluated across seven critical vulnerability classes on real-world smart contracts, it automatically generates formally verified proofs, substantially reducing manual auditing effort. Experimental results demonstrate superior accuracy and efficiency compared to baseline methods.

Technology Category

• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories
• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Knowledge Representation and Reasoning: Automated Reasoning and Theorem Proving

Application Category

• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Security and Privacy: Cryptocurrency and smart contracts
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

The high rate of false alarms from static analysis tools and Large Language Models (LLMs) complicates vulnerability detection in Solidity Smart Contracts, demanding methods that can formally or empirically prove the presence of defects. This paper introduces a novel detection pipeline that integrates custom Slither-based detectors, LLMs, Kontrol, and Forge. Our approach is designed to reliably detect defects and generate proofs. We currently perform experiments with promising results for seven types of critical defects. We demonstrate the pipeline's efficacy by presenting our findings for three vulnerabilities -- Reentrancy, Complex Fallback, and Faulty Access Control Policies -- that are challenging for current verification solutions, which often generate false alarms or fail to detect them entirely. We highlight the potential of either symbolic or concrete execution in correctly classifying such code faults. By chaining these instruments, our method effectively validates true positives, significantly reducing the manual verification burden. Although we identify potential limitations, such as the inconsistency and the cost of LLMs, our findings establish a robust framework for combining heuristic analysis with formal verification to achieve more reliable and automated smart contract auditing.

Problem

• Reducing false alarms in Solidity smart contract vulnerability detection
• Validating true positives through symbolic and concrete execution
• Automating defect proof generation for critical security vulnerabilities

Innovation

• Combining symbolic and concrete execution techniques
• Integrating custom Slither detectors with LLMs
• Chaining heuristic analysis with formal verification