SLURM Heterogeneous Jobs for Hybrid Classical-Quantum Workflows

2025-06-04

AI Summary

Hybrid classical-quantum workflows suffer from high quantum resource idle time and prolonged end-to-end execution latency. Method: This paper proposes a lightweight co-scheduling approach leveraging SLURM’s native heterogeneous job mechanism—requiring no kernel modification or dedicated scheduler deployment. It achieves tight coupling between classical computation and quantum hardware invocation via modular workflow decomposition, explicit dataflow management, and shell-script-based orchestration. Contribution/Results: To our knowledge, this is the first systematic extension of SLURM’s heterogeneous job capabilities to quantum computing. The method significantly reduces quantum device idle time and overall wall-clock execution time. Experimental evaluation demonstrates that the solution is both concise and efficient, enabling plug-and-play integration of quantum computing capabilities into existing HPC centers. It provides a practical, low-overhead pathway for supercomputing facilities to rapidly adopt quantum acceleration without infrastructure overhaul.

Technology Category

• Machine Learning: Quantum Machine Learning
• Planning, Routing, and Scheduling: Learning for Planning and Scheduling
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Economics, Online Markets and Human Computation: Architectures and workflows that use LLMs for crowd work
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

A method for efficient scheduling of hybrid classical-quantum workflows is presented, based on standard tools available on common supercomputer systems. Moderate interventions by the user are required, such as splitting a monolithic workflow in to basic building blocks and ensuring the data flow. This bares the potential to significantly reduce idle time of the quantum resource as well as overall wall time of co-scheduled workflows. Relevant pseudo-code samples and scripts are provided to demonstrate the simplicity and working principles of the method.

Problem

• Scheduling hybrid classical-quantum workflows efficiently
• Reducing idle time of quantum resources
• Minimizing wall time of co-scheduled workflows

Innovation

• SLURM schedules hybrid classical-quantum workflows
• User splits workflow into basic blocks
• Reduces quantum resource idle time