NumScout: Unveiling Numerical Defects in Smart Contracts using LLM-Pruning Symbolic Execution

2025-03-13

AI Summary

This paper addresses previously unstudied numerical defects—including precision loss and rounding bias—in high-TVL Ethereum smart contracts. We propose the first automated detection framework covering five novel defect classes. Methodologically, we formally define and empirically validate all five defect types; introduce an LLM-driven function pruning mechanism to enhance target focus and efficiency of symbolic execution; and integrate source-code and bytecode analysis via pattern-guided symbolic execution coupled with LLM-assisted function relevance inference. Evaluated on 6,617 real-world contracts, our framework identifies 1,774 defect instances with an overall precision of 89.7%, achieving a 28.4% speedup over state-of-the-art tools—demonstrating significant improvements in both accuracy and scalability.

Technology Category

• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy
• Machine Learning: Ethics, Bias, and Fairness
• Search and Optimization: Evaluation and Analysis

Application Category

• Security and Privacy: Cryptocurrency and smart contracts
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

In recent years, the Ethereum platform has witnessed a proliferation of smart contracts, accompanied by exponential growth in total value locked (TVL). High-TVL smart contracts often require complex numerical computations, particularly in mathematical financial models used by many decentralized applications (DApps). Improper calculations can introduce numerical defects, posing potential security risks. Existing research primarily focuses on traditional numerical defects like integer overflow, and there is currently a lack of systematic research and effective detection methods targeting new types of numerical defects. In this paper, we identify five new types of numerical defects through the analysis of 1,199 audit reports by utilizing the open card method. Each defect is defined and illustrated with a code example to highlight its features and potential consequences. We also propose NumScout, a symbolic execution-based tool designed to detect these five defects. Specifically, the tool combines information from source code and bytecode, analyzing key operations such as comparisons and transfers, to effectively locate defects and report them based on predefined detection patterns. Furthermore, NumScout uses a large language model (LLM) to prune functions which are unrelated to numerical operations. This step allows symbolic execution to quickly enter the target function and improve runtime speed by 28.4%. We run NumScout on 6,617 real-world contracts and evaluated its performance based on manually labeled results. We find that 1,774 contracts contained at least one of the five defects, and the tool achieved an overall precision of 89.7%.

Problem

• Detects new numerical defects in Ethereum smart contracts.
• Improves detection speed using LLM-pruning symbolic execution.
• Evaluates effectiveness on real-world contracts with high precision.

Innovation

• Symbolic execution detects numerical defects in contracts.
• LLM prunes functions to speed up execution by 28.4%.
• Combines source code and bytecode for precise defect detection.