EPOCH: Enabling Preemption Operation for Context Saving in Heterogeneous FPGA Systems

2025-01-27

AI Summary

To address context loss due to task preemption and the lack of fine-grained state preservation and restoration mechanisms in multi-tenant cloud FPGAs, this paper proposes the first plug-and-play, hardware-aware runtime framework. The framework enables cycle-accurate freezing and restoration of the full hardware state—including LUTs, FFs, BRAMs, and DSPs—at arbitrary clock cycles, and introduces a novel cross-logic-domain unified clock synchronization mechanism to eliminate timing violations. Implemented on the Zynq-7000 SoC, it integrates register-level state capture, off-chip memory snapshot storage, and clock-domain alignment. Experimental results show that single-frame context save and restore latencies are 62.2 μs and 67.4 μs, respectively, with no task restart required. This significantly enhances real-time responsiveness, hardware isolation, and scheduling flexibility in multi-tenant FPGA-as-a-Service environments.

Technology Category

• Machine Learning: Hardware-aware ML
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Computer Vision: Multi-modal Vision

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Location- and context-aware Web and WoT applications and services
• Search and Retrieval-Augmented AI: Personalized, context-aware and across-device search
• Security and Privacy: Data transparency and provenance

Abstract

FPGAs are increasingly used in multi-tenant cloud environments to offload compute-intensive tasks from the main CPU. The operating system (OS) plays a vital role in identifying tasks suitable for offloading and coordinating between the CPU and FPGA for seamless task execution. The OS leverages preemption to manage CPU efficiently and balance CPU time; however, preempting tasks running on FPGAs without context loss remains challenging. Despite growing reliance on FPGAs, vendors have yet to deliver a solution that fully preserves and restores task context. This paper presents EPOCH, the first out-of-the-box framework to seamlessly preserve the state of tasks running on multi-tenant cloud FPGAs. EPOCH enables interrupting a tenant's execution at any arbitrary clock cycle, capturing its state, and saving this 'state snapshot' in off-chip memory with fine-grain granularity. Subsequently, when task resumption is required, EPOCH can resume execution from the saved 'state snapshot', eliminating the need to restart the task from scratch. EPOCH automates intricate processes, shields users from complexities, and synchronizes all underlying logic in a common clock domain, mitigating timing violations and ensuring seamless handling of interruptions. EPOCH proficiently captures the state of fundamental FPGA elements, such as look-up tables, flip-flops, block--RAMs, and digital signal processing units. On real hardware, ZynQ-XC7Z020 SoC, the proposed solution achieves context save and restore operations per frame in 62.2us and 67.4us, respectively.

Problem

• Cloud Computing
• FPGA
• Task Switching

Innovation

• EPOCH
• FPGA State Preservation
• Efficient Task Switching