Combining Type Inference and Automated Unit Test Generation for Python

2025-07-02

AI Summary

Dynamic-typed languages like Python lack static type information, hindering automated unit test generation. To address this, this paper proposes a runtime type-tracing–based test generation approach: during program execution, it dynamically observes function invocations to incrementally infer argument and return types, then feeds these inferred types back into the test generation process to guide test case construction. Implemented within the Pynguin framework, the method integrates dynamic analysis with type inference, establishing— for the first time—the closed-loop co-adaptation of type inference and test generation. Experimental results show that the approach improves branch coverage by up to 90.0%, significantly boosts mutation scores, and achieves type inference quality comparable to state-of-the-art static analyzers (e.g., Pyright, pytype). This work introduces a novel paradigm for automated testing of dynamic languages.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Reasoning under Uncertainty: Probabilistic Programming
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications

Abstract

Automated unit test generation is an established research field that has so far focused on statically-typed programming languages. The lack of type information in dynamically-typed programming languages, such as Python, inhibits test generators, which heavily rely on information about parameter and return types of functions to select suitable arguments when constructing test cases. Since automated test generators inherently rely on frequent execution of candidate tests, we make use of these frequent executions to address this problem by introducing type tracing, which extracts type-related information during execution and gradually refines the available type information. We implement type tracing as an extension of the Pynguin test-generation framework for Python, allowing it (i) to infer parameter types by observing how parameters are used during runtime, (ii) to record the types of values that function calls return, and (iii) to use this type information to increase code coverage. The approach leads to up to 90.0% more branch coverage, improved mutation scores, and to type information of similar quality to that produced by other state-of-the-art type-inference tools.

Problem

• Lack of type information in Python inhibits automated test generation
• Type tracing extracts and refines type data during test execution
• Improving test coverage and mutation scores via inferred type information

Innovation

• Type tracing extracts runtime type information
• Pynguin extension infers parameter and return types
• Improved test coverage and mutation scores