rdma

In large-scale distributed machine learning (ML), collective communication suffers from high tail latency, while existing RDMA protocols (e.g., RoCE) exhibit poor scalability due to stringent reliability and in-order delivery guarantees. Method: This paper proposes a lightweight RDMA architecture tailored to ML workloads—intentionally relaxing strict reliability and ordering requirements to exploit ML’s inherent tolerance to partial packet loss. Retransmission and error correction are offloaded to the upper-layer training pipeline; the design employs best-effort transmission, adaptive timeout, priority-based data scheduling, retains DCQCN for congestion control, and integrates Hadamard transform for efficient fault tolerance. Contribution/Results: Experiments show a 2.3× reduction in 99th-percentile latency, 67% reduction in BRAM resource usage, and nearly 2× improvement in NIC fault recovery time—significantly enhancing scalability and resilience of ML communication in ten-thousand-GPU clusters.

High-frequency, fine-grained I/O in AI training imposes severe performance bottlenecks on TCP-based host-mediated storage stacks. Method: This paper proposes a POSIX-compliant, RDMA-first object storage system that fully offloads the DAOS client onto NVIDIA BlueField-3 SmartNICs—enabling host bypass and user-space storage stack offload. It adopts a decoupled architecture: a gRPC-based control plane and a UCX/libfabric-based data plane, supporting multi-tenancy isolation and inline services on the DPU. Contribution/Results: Experiments demonstrate significantly higher RDMA throughput than TCP across both small and large I/O workloads. End-to-end performance matches that of direct host-attached storage while preserving RDMA’s low latency and high bandwidth. This work presents the first validation of RDMA feasibility and scalability under full client-side offload, establishing an efficient, scalable data path foundation for GPU-centric large-model training.

Distributed ML at scale (thousands of GPUs) faces a critical bottleneck: tail latency in collective communications. Conventional RDMA solutions (e.g., RoCE) rely heavily on retransmission and strict in-order delivery, resulting in high protocol complexity, tail-latency sensitivity, and pipeline stalls. This work proposes a domain-specific RDMA NIC architecture tailored for ML workloads, the first to explicitly exploit ML’s intrinsic tolerance to packet loss and out-of-order delivery. It eliminates retransmission and rigid ordering, instead adopting adaptive timeout–driven best-effort, out-of-order transmission. Reliability recovery is delegated to the ML layer—via techniques such as Hadamard transforms and erasure coding—enabling a lightweight, resilient NIC design. The architecture maintains full compatibility with standard congestion control mechanisms (DCQCN, EQDS, Swift). Evaluations on Hyperstack and CloudLab demonstrate 2.0× speedup in training time-to-accuracy (TTA), 1.6× higher inference throughput, 3.5× reduction in 99th-percentile latency, 2.7× lower BRAM utilization, and nearly doubled fault tolerance.

Datacenter networks have become a performance bottleneck for data-intensive applications such as machine learning, and existing RDMA protocols struggle to meet the stringent requirements of SmartNICs and accelerators. This paper proposes BALBOA: a service-enhanced RDMA architecture tailored for SmartNICs. Its core is an open-source, customizable RoCE v2–compatible protocol stack capable of supporting hundreds of queue pairs and sustaining 100 Gbps line-rate processing. BALBOA innovatively integrates host-bypass, pipelined compute offloading, and in-network computation, unifying protocol processing and data acceleration on FPGA. It further extends functionality with hardware-accelerated encryption, ML packet classification, and line-rate data preprocessing. Experimental evaluation on an FPGA-based cluster demonstrates that BALBOA achieves latency and throughput comparable to commercial NICs, enabling end-to-end line-rate offloading for recommendation systems. Results validate its high performance, ultra-low latency, and adaptability across diverse application scenarios.

RDMA offers low latency and high throughput but suffers from a weak memory model lacking formal foundations, hindering the construction of correct, composable multi-node concurrent objects. Method: We introduce LOCO—the first RDMA-native multi-node object library supporting formal verification—bridging shared-memory and distributed programming paradigms. To address RDMA consistency modeling challenges, we design Mowgli, a modular, declarative verification framework that decouples proof logic from the memory model and enables systematic correctness proofs leveraging locality properties. Contribution/Results: LOCO achieves performance comparable to custom RDMA systems while drastically simplifying the programming model. All core components are fully formally verified. LOCO provides a high-reliability, high-performance infrastructure foundation for data centers, HPC, and AI/ML workloads.

This work addresses the challenges of deploying hash tables in RDMA-based remote memory under IoT big data scenarios, where limited local memory necessitates offloading to disaggregated memory systems. The key obstacles include frequent remote accesses, ineffective concurrency control, and constrained RDMA resources. To tackle these issues, the study presents the first holistic design that jointly optimizes hash collision resolution, concurrency mechanisms, and RDMA hardware characteristics. By leveraging one-sided RDMA operations, lightweight concurrency control, and careful structuring of NIC resources—including queues, atomic operations, and memory registration—the proposed paradigm significantly improves latency, throughput, and scalability of distributed key-value systems in memory-disaggregated architectures.

This work addresses the security vulnerabilities of Remote Direct Memory Access (RDMA) in untrusted environments, where its CPU-bypassing nature circumvents conventional security mechanisms. The authors present the first native RDMA encryption implementation directly within the data plane of a programmable Tofino switch, leveraging the P4 language to integrate AES-128 encryption and offload cryptographic operations from the host CPU to the network data plane. This approach ensures strong security without compromising performance. Experimental results demonstrate sustained throughput ranging from 0.37 to 1.9 Gbps for packet sizes between 16 and 128 bytes, confirming the feasibility of simultaneously achieving high performance and robust security. The proposed method overcomes the longstanding challenge of adapting traditional security mechanisms to high-speed RDMA communication.

This work addresses the high memory overhead and latency in modern data centers caused by the InfiniBand queue-pair abstraction of RDMA, which creates a performance bottleneck at the NIC. The authors present the first clean-room, open-source implementation of Huawei’s Unified Bus (UB) protocol, encompassing both transport and transaction layers. UB decouples application endpoints from host transport state, supports on-demand ordering, and enables remote memory access using native CPU load/store semantics. The project provides reproducible evaluation platforms at RTL, SystemC, and gem5 levels, enabling fair comparison with RoCEv2. Experimental results demonstrate that, for 64-byte remote reads, UB achieves an end-to-end latency of approximately 500 nanoseconds—4.37× lower than RoCEv2—with 2.80× higher throughput, while consuming only 14% of the logic resources on an Alveo U50 FPGA.

This work addresses the inefficiencies of existing fault-tolerance mechanisms for RDMA link failures, which uniformly retransmit all in-flight requests upon connection disruption, leading to wasted bandwidth, semantic errors, and high recovery overhead. The authors propose a failure-type-aware recovery mechanism that leverages a lightweight completion log to track the execution status of each request, thereby distinguishing between already executed and unexecuted operations. By selectively retransmitting only necessary requests and restoring their results, this approach achieves precise, execution-state-based retransmission for the first time. It prevents redundant execution of non-idempotent operations, ensures transactional consistency, and eliminates the need for connection reestablishment. Experimental results demonstrate negligible memory overhead, a steady-state latency penalty of merely 0.6–10%, and a 65% reduction in recovery retransmission time.

Existing one-sided communication frameworks, such as MPI RMA and OpenSHMEM, struggle to meet the demands of high-performance computing due to limited scalability, rigid memory models, and the requirement for blocking synchronization during window creation. To address these limitations, this work proposes RAMC—a lightweight explicit one-sided communication library built upon persistent one-way communication channels. Leveraging the hardware capabilities of HPE Slingshot interconnects and employing memory region counters for efficient completion notification, RAMC preserves the flexibility of RDMA while overcoming constraints inherent in traditional shared-memory models. Implemented atop the libfabric interface, RAMC demonstrates robust scalability up to 250 nodes (196,000 processes), achieving bandwidth improvements of 100%–130% over Cray MPI on libfabric 1.15.2 and 30%–45% on libfabric 2.3.1.