🤖 AI Summary
This work addresses the challenge of reliably storing volatile data in MPI applications operating in environments prone to frequent hardware failures. The authors design and implement an elastic key-value store built upon MPI Remote Memory Access (RMA), leveraging MPI’s passive-target RMA for efficient one-sided data access. Fault detection is achieved through integration with Open MPI’s User-Level Failure Mitigation (ULFM) extension, while data and execution recovery are realized via redundant replicas maintained on surviving processes. This study presents the first systematic exploration of combining MPI RMA with ULFM for fault-tolerant storage, identifies critical missing features in current ULFM support for RMA, and proposes practical workarounds. Experimental evaluation of the prototype demonstrates the effectiveness and practicality of the proposed approach under realistic failure scenarios.
📝 Abstract
As hardware failures such as node losses become increasingly common, MPI programmers may want to save vulnerable data in a resilient store. While third-party storage solutions such as Redis or the Hazelcast IMap exist, a tailored, MPI-based store may be easier to integrate and can be optimized for particular application needs.
This paper considers the implementation of such a store, which is intended as a component in a resilient task-based runtime system written in MPI. The store holds redundant data copies as key-value pairs in the main memories of multiple processes. Since store access operations, such as reads and writes, are naturally one-sided, we implemented the store with passive target MPI RMA functions. Process aborts are detected with the user-level failure mitigation (ULFM) extension of Open MPI. After failures, the program recovers on the surviving processes and continues with the intact data copies.
Our implementation proved difficult, since several proposed ULFM functionalities for RMA have not yet been implemented. Even assuming their existence, we think that the programming task could be simplified. This paper describes our experiences, lists functionalities that we missed, and explains a workaround that we adopted in our implementation.