A Python workflow definition for computational materials design

2025-05-26

AI Summary

In computational materials science, multi-source workflow management systems (e.g., AiiDA, jobflow, pyiron) employ heterogeneous formats, severely hindering workflow interoperability and FAIR reproducibility. To address this, we propose Python Workflow Definition (PWD)—the first Python-native, cross-platform workflow exchange standard tailored for materials design. PWD decouples scientific logic from execution environments by encapsulating workflows via three lightweight, portable, and re-executable components: conda environments, Python function modules, and JSON-encoded directed acyclic graphs (DAGs). It enables bidirectional import/export among the three major systems, supports parameter tuning and resource reconfiguration, and is openly integrated into each platform. This work establishes the first domain-specific, unified workflow exchange paradigm, significantly enhancing interoperability, reproducibility, and sustainable sharing of computational materials workflows.

Technology Category

• Planning, Routing, and Scheduling: Mixed Discrete/Continuous Planning
• Machine Learning: Hardware-aware ML
• Cognitive Modeling & Cognitive Systems: Computational Creativity

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Sustainability and carbon-aware systems for Web, mobile, and WoT
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Economics, Online Markets and Human Computation: Architectures and workflows that use LLMs for crowd work

Abstract

Numerous Workflow Management Systems (WfMS) have been developed in the field of computational materials science with different workflow formats, hindering interoperability and reproducibility of workflows in the field. To address this challenge, we introduce here the Python Workflow Definition (PWD) as a workflow exchange format to share workflows between Python-based WfMS, currently AiiDA, jobflow, and pyiron. This development is motivated by the similarity of these three Python-based WfMS, that represent the different workflow steps and data transferred between them as nodes and edges in a graph. With the PWD, we aim at fostering the interoperability and reproducibility between the different WfMS in the context of Findable, Accessible, Interoperable, Reusable (FAIR) workflows. To separate the scientific from the technical complexity, the PWD consists of three components: (1) a conda environment that specifies the software dependencies, (2) a Python module that contains the Python functions represented as nodes in the workflow graph, and (3) a workflow graph stored in the JavaScript Object Notation (JSON). The first version of the PWD supports directed acyclic graph (DAG)-based workflows. Thus, any DAG-based workflow defined in one of the three WfMS can be exported to the PWD and afterwards imported from the PWD to one of the other WfMS. After the import, the input parameters of the workflow can be adjusted and computing resources can be assigned to the workflow, before it is executed with the selected WfMS. This import from and export to the PWD is enabled by the PWD Python library that implements the PWD in AiiDA, jobflow, and pyiron.

Problem

• Lack of interoperability among computational materials science workflows
• Need for reproducible workflow exchange between Python-based WfMS
• Separation of scientific and technical complexity in workflow design

Innovation

• Python Workflow Definition for interoperability
• Three components: conda, Python module, JSON
• Supports DAG-based workflows across WfMS