SymGPT: Auditing Smart Contracts via Combining Symbolic Execution with Large Language Models

2025-02-11

AI Summary

Existing ERC-compliance verification methods—manual auditing, static analysis tools, and LLMs—suffer from low efficiency and insufficient accuracy in detecting rule violations. Method: This paper proposes an LLM-driven symbolic verification framework that uniquely integrates instruction-tuned large language models with EBNF-based grammar modeling to automatically translate natural-language ERC specifications into executable constraints; it further combines symbolic execution (via Mythril/Foundry) for formal violation detection and interpretable counterexample path generation. Contribution/Results: We conduct the first empirical analysis of all 132 ERC rules, establishing the first end-to-end executable audit framework. Evaluated on 4,000 real-world Ethereum smart contracts, our system identifies 5,783 violations, including 1,375 linked to explicit asset-theft paths. It significantly outperforms six state-of-the-art tools and professional manual auditing services in precision, recall, and scalability.

Technology Category

• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories
• Knowledge Representation and Reasoning: Automated Reasoning and Theorem Proving

Application Category

• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

To govern smart contracts running on Ethereum, multiple Ethereum Request for Comment (ERC) standards have been developed, each having a set of rules to guide the behaviors of smart contracts. Violating the ERC rules could cause serious security issues and financial loss, signifying the importance of verifying smart contracts follow ERCs. Today's practices of such verification are to manually audit each single contract, use expert-developed program-analysis tools, or use large language models (LLMs), all of which are far from effective in identifying ERC rule violations. This paper introduces SymGPT, a tool that combines the natural language understanding of large language models (LLMs) with the formal guarantees of symbolic execution to automatically verify smart contracts' compliance with ERC rules. To develop SymGPT, we conduct an empirical study of 132 ERC rules from three widely used ERC standards, examining their content, security implications, and natural language descriptions. Based on this study, we design SymGPT by first instructing an LLM to translate ERC rules into a defined EBNF grammar. We then synthesize constraints from the formalized rules to represent scenarios where violations may occur and use symbolic execution to detect them. Our evaluation shows that SymGPT identifies 5,783 ERC rule violations in 4,000 real-world contracts, including 1,375 violations with clear attack paths for stealing financial assets, demonstrating its effectiveness. Furthermore, SymGPT outperforms six automated techniques and a security-expert auditing service, underscoring its superiority over current smart contract analysis methods.

Problem

• Automates verification of Ethereum smart contracts compliance.
• Combines LLMs with symbolic execution for ERC rule checks.
• Identifies and mitigates ERC rule violations effectively.

Innovation

• Combines LLMs with symbolic execution
• Automates ERC rule compliance verification
• Translates ERC rules into EBNF grammar