🤖 AI Summary
This work addresses the widespread misuse of statistical methods—often stemming from implicit or ambiguous assumptions—which exacerbates the reproducibility crisis in scientific research, particularly in hypothesis testing and meta-analysis where formal verification mechanisms are lacking. To bridge this gap, the authors propose the first formal verification framework tailored for Python-based statistical programs. By developing a Why3-py frontend, they translate dynamically typed, runtime-polymorphic Python code into the WhyML intermediate representation and extend the StatWhy tool to support meta-analysis verification. Integrating program transformation, static analysis, and formal verification techniques, this approach enables, for the first time, automated correctness verification of statistical programs written in Python, effectively uncovering overlooked assumptions and misuses, thereby filling a critical void in the formal verification of statistical software.
📝 Abstract
The reproducibility crisis in scientific research has received widespread recognition, thereby increasing the importance of meta-analyses that integrate statistical analyses from multiple studies. However, statistical methods often have ambiguous and implicit underlying assumptions, which can lead to their erroneous applications and interpretations. To address this issue, we propose a formal verification framework for statistical programs written in Python. Specifically, we present Why3-py, a Python front-end for the Why3 verification platform that transforms Python programs into verification-oriented WhyML representations suitable for formal verification, addressing the challenges arising from Python's dynamic typing and runtime polymorphism. Furthermore, we extend the StatWhy tool to support the verification of meta-analysis methods. These tools enable users to identify overlooked assumptions and misuse of analyses, and to verify the correctness of Python programs for hypothesis testing and for meta-analyses.