A Systematic Evaluation of the Potential of Carbon-Aware Execution for Scientific Workflows

2025-08-20

AI Summary

Scientific workflows are computationally intensive, latency-tolerant, and highly scalable, yet their long-running executions on clusters incur substantial carbon emissions. Existing carbon-aware computing approaches lack workflow-specific design. This paper presents the first systematic study of carbon-aware scientific workflow execution. We model carbon footprint using both average and marginal carbon intensity, and integrate our approach with the Nextflow engine to propose a multi-dimensional optimization strategy comprising time-shifting (scheduling tasks according to grid carbon intensity fluctuations), task-level suspend/resume, and heterogeneous resource autoscaling. Experiments in representative cluster environments demonstrate that time-shifting reduces carbon emissions by up to 82.3%, while resource autoscaling achieves up to 67.1% reduction; the combined strategy significantly enhances the carbon efficiency of scientific computing. Our work establishes a scalable methodology and empirical foundation for carbon-aware high-performance computing.

Technology Category

• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Machine Learning: Hardware-aware ML
• Search and Optimization: Combinatorial Optimization

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Sustainability and carbon-aware systems for Web, mobile, and WoT
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Scientific workflows are widely used to automate scientific data analysis and often involve computationally intensive processing of large datasets on compute clusters. As such, their execution tends to be long-running and resource-intensive, resulting in substantial energy consumption and, depending on the energy mix, carbon emissions. Meanwhile, a wealth of carbon-aware computing methods have been proposed, yet little work has focused specifically on scientific workflows, even though they present a substantial opportunity for carbon-aware computing because they are often significantly delay tolerant, efficiently interruptible, highly scalable and widely heterogeneous. In this study, we first exemplify the problem of carbon emissions associated with running scientific workflows, and then show the potential for carbon-aware workflow execution. For this, we estimate the carbon footprint of seven real-world Nextflow workflows executed on different cluster infrastructures using both average and marginal carbon intensity data. Furthermore, we systematically evaluate the impact of carbon-aware temporal shifting, and the pausing and resuming of the workflow. Moreover, we apply resource scaling to workflows and workflow tasks. Finally, we report the potential reduction in overall carbon emissions, with temporal shifting capable of decreasing emissions by over 80%, and resource scaling capable of decreasing emissions by 67%.

Problem

• Evaluating carbon emissions from scientific workflow executions
• Assessing carbon-aware methods for reducing workflow energy use
• Quantifying potential emission reductions via temporal and resource adjustments

Innovation

• Carbon-aware temporal shifting for workflows
• Pausing and resuming workflow execution strategically
• Resource scaling for workflow tasks optimization