🤖 AI Summary
This work addresses the challenge of graph-based approximate nearest neighbor search (Graph ANNS) when index sizes exceed the memory capacity of a single GPU, a scenario where existing CPU-centric remote I/O approaches struggle to meet the dynamic and irregular data access patterns inherent in GPU-driven graph traversal. To overcome this limitation, the authors propose the first GPU-centric NVMe-oF remote I/O architecture, which fully offloads query execution, page fault handling, and recovery decisions to the GPU, relegating the CPU to a mere transport proxy. They further introduce a staged remote operation mechanism coupled with an ANNS-specific scheduling policy to effectively overlap graph traversal with remote page loading. Experimental results on the SIFT1M DiskANN workload demonstrate that the proposed system achieves 1.31× speedup over the state-of-the-art remote I/O solution and 3.73× speedup compared to a direct-attached remote page caching approach.
📝 Abstract
Graph-based approximate nearest neighbor search (ANNS) is increasingly used in vector databases and retrieval-augmented generation services, but large vector indexes often exceed the memory capacity of a single GPU server. NVMe over Fabrics (NVMe-oF) provides an attractive storage-disaggregation substrate, yet existing remote storage paths are still largely CPU-centered: the CPU forms I/O requests, drives transport progress, and determines when GPU computation can resume. This organization is poorly matched to graph ANNS, where the next data access is discovered inside GPU graph traversal.
This paper presents GORIO, a system study that extends GPU-centered local I/O to remote storage and specializes the resulting substrate for graph ANNS over NVMe-oF. GORIO keeps query evolution, page-miss generation, pending-query state, and resume decisions on the GPU, while the CPU acts only as an NVMe-oF transport and completion proxy. The design has two layers: a GPU-direct remote I/O path that turns local page-cache misses into split-phase remote operations, and ANNS-specific scheduling mechanisms that overlap graph traversal with remote page service. On a SIFT1M DiskANN-style graph workload over an RDMA NVMe-oF path, GORIO is 1.31$\times$ faster than the state-of-the-art remote-I/O reference path and 3.73$\times$ faster than the direct remote page-cache path. These results demonstrate a concrete GPU-centered remote I/O substrate for graph ANNS.