Toward a Unified GPU-Aware OpenSHMEM Specification

2026-07-08

AI Summary

This work addresses the lack of portable memory semantics for GPU architectures in existing OpenSHMEM implementations, which has led to fragmented vendor-specific approaches. To resolve this, the authors propose a lightweight, backward-compatible auxiliary specification that, for the first time, defines a vendor-neutral GPU-scoped memory space abstraction along with a complementary capability query mechanism. This unified framework harmonizes the semantics of remote memory access (RMA), atomic operations, synchronization, and collective operations under both host-initiated and device-initiated execution models. Built upon the Partitioned Global Address Space (PGAS) paradigm, the proposal is accompanied by a prototype implementation deployable across multiple GPU vendors, thereby establishing the first practical foundation and standardization pathway for GPU extensions to the OpenSHMEM specification.

Abstract

Leadership-class HPC systems are now accelerator-centric, with GPUs providing most floating-point throughput and memory bandwidth. As next-generation systems increasingly integrate accelerators through high-speed memory fabrics and system interconnects, exposing larger tightly coupled device domains, \ac{PGAS} models such as OpenSHMEM provide a natural abstraction for expressing fine-grained remote memory operations across these devices. While OpenSHMEM 1.x offers a lean PGAS model for irregular communication, atomics, fine-grained synchronization, and collectives, its memory model lacks portable semantics for accelerator architectures. As a result, existing GPU-enabled OpenSHMEM implementations differ in memory management, capability discovery, and operation semantics, limiting portability and ecosystem cohesion. This risks fracturing the community that OpenSHMEM was originally created to unify. This paper proposes an OpenSHMEM Auxiliary Specification for GPU-Aware Communication, designed as a lightweight, backward-compatible extension to OpenSHMEM 1.x. The auxiliary specification introduces a minimal memory model extension via a GPU-scoped memory space abstraction, along with capability queries and well-defined semantics for using \acs{GPU}-attached buffers in RMA, atomic, synchronization, and collective operations. This is initially conceived through the lens of a host-initiated interface, although it provides a general set of semantics that also allow for optional device-initiated support. A central goal of this effort is to demonstrate that GPU-aware OpenSHMEM semantics can be specified and implemented across GPUs from multiple vendors, providing a practical and rapidly implementable step toward unification under a vendor-neutral specification while informing the design of future OpenSHMEM specifications.

Problem

• OpenSHMEM
• GPU-aware
• PGAS
• memory model
• portability

Innovation

• GPU-aware
• OpenSHMEM
• PGAS
• memory model
• portability