Solsmith: Solidity Random Program Generator for Compiler Testing

2025-06-04

AI Summary

To address challenges in verifying Solidity compiler reliability—including difficulty in test case generation, low coverage, and frequent triggering of undefined behavior—this paper proposes the first random program generation method tailored to Solidity’s language features. Our approach integrates (1) syntax-driven structured generation, (2) constraint-guided semantic validity enforcement—specifically avoiding undefined behavior and mitigating IR-level behavioral discrepancies—and (3) a differential execution–based validation framework. We further introduce a novel high-frequency optimization enabling strategy to enhance compiler path coverage. Experimental results demonstrate that the generated smart contracts exhibit strong syntactic and semantic validity, high structural diversity, and significantly improved branch coverage over baseline methods. Crucially, our technique uncovered four previously unknown compiler defects, all confirmed by the official Solidity team—thereby validating its effectiveness in both defect detection capability and practical utility.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Security and Privacy: Cryptocurrency and smart contracts
• Web Mining and Content Analysis: Web data generation and simulation
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

Smart contracts are computer programs that run on blockchain platforms, with Solidity being the most widely used language for their development. As blockchain technology advances, smart contracts have become increasingly important across various fields. In order for smart contracts to operate correctly, the correctness of the compiler is particularly crucial. Although some research efforts have been devoted to testing Solidity compilers, they primarily focus on testing methods and do not address the core issue of generating test programs. To fill this gap, this paper designs and implements Solsmith, a test program generator specifically aimed at uncovering defects in Solidity compilers. It tests the compiler correctness by generating valid and diverse Solidity programs. We have designed a series of unique program generation strategies tailored to Solidity, including enabling optimizations more frequently, avoiding undefined behaviour, and mitigating behavioural differences caused by intermediate representations. To validate the effectiveness of Solsmith, we assess the effectiveness of the test programs generated by Solsmith using the approach of differential testing. The preliminary results show that Solsmith can generate the expected test programs and uncover four confirmed defects in Solidity compilers, demonstrating the effectiveness and potential of Solsmith.

Problem

• Generating diverse Solidity programs for compiler testing
• Uncovering defects in Solidity compilers effectively
• Validating compiler correctness through differential testing

Innovation

• Generates diverse Solidity test programs
• Uses unique Solidity-specific generation strategies
• Applies differential testing for validation