Designing FAIR Workflows at OLCF: Building Scalable and Reusable Ecosystems for HPC Science

2025-11-01

AI Summary

HPC users frequently develop center-specific, customized digital artifacts, leading to redundant development across users and disciplines; existing FAIR practices remain siloed within domains and thus fail to support cross-disciplinary HPC collaboration. Method: We propose a novel FAIRization paradigm centered on “reusable components,” moving beyond monolithic workflow encapsulation to enable fine-grained discovery, interoperability, and reuse of computational assets across domains. Leveraging the EOSC and EOSC-Life FAIR Workflows Collaboratory architectures, we integrate standardized metadata schemas, community-driven best practices, and modular design principles to construct the first FAIR component implementation model tailored for HPC centers. Contribution/Results: The model significantly improves sharing efficiency and reuse rates of research digital assets, reduces redundant development costs, and provides a scalable infrastructure foundation for open science and long-term value accumulation in HPC environments.

Technology Category

• Humans and AI: Other Foundations of Human Computation & AI
• Machine Learning: Hardware-aware ML
• Cognitive Modeling & Cognitive Systems: (Computational) Cognitive Architectures

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Security and Privacy: Data transparency and provenance
• Economics, Online Markets and Human Computation: Fairness, privacy, and diversity in economic environments

Abstract

High Performance Computing (HPC) centers provide advanced infrastructure that enables scientific research at extreme scale. These centers operate with hardware configurations, software environments, and security requirements that differ substantially from most users'local systems. As a result, users often develop customized digital artifacts that are tightly coupled to a given HPC center. This practice can lead to significant duplication of effort as multiple users independently create similar solutions to common problems. The FAIR Principles offer a framework to address these challenges. Initially designed to improve data stewardship, the FAIR approach has since been extended to encompass software, workflows, models, and infrastructure. By encouraging the use of rich metadata and community standards, FAIR practices aim to make digital artifacts easier to share and reuse, both within and across scientific domains. Many FAIR initiatives have emerged within individual research communities, often aligned by discipline (e.g. bioinformatics, earth sciences). These communities have made progress in adopting FAIR practices, but their domain-specific nature can lead to silos that limit broader collaboration. Thus, we propose that HPC centers play a more active role in fostering FAIR ecosystems that support research across multiple disciplines. This requires designing infrastructure that enables researchers to discover, share, and reuse computational components more effectively. Here, we build on the architecture of the European Open Science Cloud (EOSC) EOSC-Life FAIR Workflows Collaboratory to propose a model tailored to the needs of HPC. Rather than focusing on entire workflows, we emphasize the importance of making individual workflow components FAIR. This component-based approach better supports the diverse and evolving needs of HPC users while maximizing the long-term value of their work.

Problem

• Addresses duplication of effort in HPC workflows due to center-specific customizations.
• Proposes FAIR principles to enhance sharing and reuse of computational components across disciplines.
• Focuses on making individual workflow components FAIR to support diverse HPC user needs.

Innovation

• Adopting FAIR principles for HPC workflow components
• Building scalable ecosystems using component-based architecture
• Leveraging metadata and standards for cross-disciplinary reuse