Preemption-Enhanced Benchmark Suite for FPGAs

2025-11-10

AI Summary

Existing FPGA cloud scheduling research lacks standardized, preemptive benchmarking frameworks; evaluations relying on private or synthetic workloads suffer from poor comparability and irreproducibility. Method: We propose FPGA-Bench—the first open-source, scalable, preemptive FPGA benchmark suite—comprising 27 real-world applications across cryptography, AI/ML, and communications. It introduces a novel hardware-level context snapshotting and restoration mechanism, enabling fine-grained resource monitoring and seamless integration with standard scheduler interfaces. Contribution/Results: FPGA-Bench enables repeatable, quantitative evaluation of scheduling fairness, resource efficiency, and preemption overhead in multi-tenant FPGA clouds. By significantly lowering the barrier to preemption strategy validation, it has already facilitated multiple studies on scheduler–OS co-design and advances standardization efforts for FPGA cloud infrastructure evaluation.

Technology Category

• Machine Learning: Hardware-aware ML
• Planning, Routing, and Scheduling: Scheduling under Uncertainty
• Cognitive Modeling & Cognitive Systems: Agent Architectures

Application Category

• Economics, Online Markets and Human Computation: Fairness and ethical considerations in crowd work and in human-in-the-loop AI systems
• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Field-Programmable Gate Arrays (FPGAs) have become essential in cloud computing due to their reconfigurability, energy efficiency, and ability to accelerate domain-specific workloads. As FPGA adoption grows, research into task scheduling and preemption techniques has intensified. However, the field lacks a standardized benchmarking framework for consistent and reproducible evaluation. Many existing studies propose innovative scheduling or preemption mechanisms but often rely on proprietary or synthetic benchmarks, limiting generalizability and making comparison difficult. This methodical fragmentation hinders effective evaluation of scheduling strategies and preemption in multi-tenant FPGA environments. This paper presents the first open-source preemption-enabled benchmark suite for evaluating FPGA preemption strategies and testing new scheduling algorithms, without requiring users to create preemption workloads from scratch. The suite includes 27 diverse applications spanning cryptography, AI/ML, computation-intensive workloads, communication systems, and multimedia processing. Each benchmark integrates comprehensive context-saving and restoration mechanisms, facilitating reproducible research and consistent comparisons. Our suite not only simplifies testing FPGA scheduling policies but also benefits OS research by enabling the evaluation of scheduling fairness, resource allocation efficiency, and context-switching performance in multi-tenant FPGA systems, ultimately supporting the development of better operating systems and scheduling policies for FPGA-based environments. We also provide guidelines for adding new benchmarks, enabling future research to expand and refine FPGA preemption and scheduling evaluation.

Problem

• Lack standardized benchmarking for FPGA preemption and scheduling evaluation
• Existing studies use proprietary benchmarks limiting comparison generalizability
• Need reproducible framework for multi-tenant FPGA scheduling strategies

Innovation

• Open-source preemption-enabled benchmark suite for FPGAs
• Includes 27 diverse applications with context mechanisms
• Enables testing scheduling fairness and resource allocation