TestMigrationsInPy: A Dataset of Test Migrations from Unittest to Pytest

2025-04-28

IEEE Working Conference on Mining Software Repositories

AI Summary

This work addresses the lack of standardized data for migrating Python projects from unittest to pytest, a gap that has hindered the development and evaluation of automated migration tools. To bridge this gap, the authors present TestMigrationsInPy, the first structured dataset comprising 923 real-world migration instances systematically collected and annotated from historical commits in open-source projects. The dataset provides fine-grained categorization of migration patterns—such as assertions and fixtures—enabling precise analysis and comparison of transformation strategies. Publicly released, TestMigrationsInPy establishes the first benchmark for automated test migration research, facilitating quantitative evaluation of diverse migration approaches and supporting future advancements in tooling for Python testing frameworks.

Technology Category

• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty
• Knowledge Representation and Reasoning: Automated Reasoning and Theorem Proving
• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning

Application Category

• Web Mining and Content Analysis: Bridging structured and unstructured data
• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Unittest and pytest are the most popular testing frameworks in Python. Overall, pytest provides some advantages, including simpler assertion, reuse of fixtures, and interoperability. Due to such benefits, multiple projects in the Python ecosystem have migrated from unittest to pytest. To facilitate the migration, pytest can also run unittest tests, thus, the migration can happen gradually over time. However, the migration can be time-consuming and take a long time to conclude. In this context, projects would benefit from automated solutions to support the migration process. In this paper, we propose TestMigrationsInPy, a dataset of test migrations from unittest to pytest. TestMigrationsInPy contains 923 real-world migrations performed by developers. Future research proposing novel solutions to migrate frameworks in Python can rely on TestMigrationsInPy as a ground truth. Moreover, as TestMigrationsInPy includes information about the migration type (e.g., changes in assertions or fixtures), our dataset enables novel solutions to be verified effectively, for instance, from simpler assertion migrations to more complex fixture migrations. TestMigrationsInPy is publicly available at: https://github.com/altinoalvesjunior/TestMigrationsInPy.

Problem

• test migration
• unittest
• pytest
• Python testing
• dataset

Innovation

• TestMigrationsInPy
• unittest to pytest migration
• test framework migration
• Python testing
• empirical dataset