An Enumerative Embedding of the Python Type System in ACL2s

2025-07-25

AI Summary

Python’s dynamic type system and static type checkers (e.g., mypy) exhibit limitations in detecting runtime type errors, leading to undetected defects despite type annotations. Method: This paper introduces a formal reasoning approach based on ACL2s, embedding an enumerable subset of Python’s type system into ACL2s using the defdata framework to define types and synthesize type-aware fuzzers. The method integrates static type parsing, formal modeling, and black-box function testing to automatically generate diverse, type-signature-compliant inputs. Contribution/Results: Evaluated on four major open-source projects, the technique achieves 68–80% average code coverage, identifies coverage-limiting control-flow patterns, and uncovers deep-seated runtime errors missed by existing tools—including those undetected by mypy. It demonstrates the efficacy and novelty of formal-methods-driven fuzz testing for validating Python type safety.

Technology Category

• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty
• Search and Optimization: Metareasoning and Metaheuristics

Application Category

• Security and Privacy: Large-scale security measurements
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Python is a high-level interpreted language that has become an industry standard in a wide variety of applications. In this paper, we take a first step towards using ACL2s to reason about Python code by developing an embedding of a subset of the Python type system in ACL2s. The subset of Python types we support includes many of the most commonly used type annotations as well as user-defined types comprised of supported types. We provide ACL2s definitions of these types, as well as defdata enumerators that are customized to provide code coverage and identify errors in Python programs. Using the ACL2s embedding, we can generate instances of types that can then be used as inputs to fuzz Python programs, which allows us to identify bugs in Python code that are not detected by state-of-the-art Python type checkers. We evaluate our work against four open-source repositories, extracting their type information and generating inputs for fuzzing functions with type signatures that are in the supported subset of Python types. Note that we only use the type signatures of functions to generate inputs and treat the bodies of functions as black boxes. We measure code coverage, which ranges from about 68% to more than 80%, and identify code patterns that hinder coverage such as complex branch conditions and external file system dependencies. We conclude with a discussion of the results and recommendations for future work.

Problem

• Embed Python type system in ACL2s for reasoning
• Generate type instances to fuzz Python programs
• Evaluate coverage and identify bugs via type signatures

Innovation

• Embed Python type system in ACL2s
• Generate type instances for fuzzing
• Measure code coverage via type signatures