Accelerating Graph-based Vector Search via Delayed-Synchronization Traversal

2024-06-18

arXiv.org

AI Summary

To address the low-latency requirements of Graph Vector Search (GVS) in large language models, search engines, and recommendation systems, this work proposes a hardware-algorithm co-design framework. First, we introduce Delayed-Synchronized Traversal (DST), a novel graph traversal algorithm that relaxes strict temporal ordering constraints to better exploit hardware parallelism. Second, we design Falcon, a domain-specific FPGA accelerator featuring on-chip Bloom filters and memory-access compression to drastically reduce memory traffic overhead. Evaluated across diverse real-world graph datasets, Falcon achieves up to 4.3× and 19.5× end-to-end latency reduction over CPU- and GPU-based baselines, respectively, while delivering 8.0× and 26.9× improvements in energy efficiency. The framework demonstrates strong performance, broad generalizability across graph structures and workloads, and tight hardware-software co-adaptation.

Technology Category

• Search and Optimization: Distributed Search
• Machine Learning: Hardware-aware ML
• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Vector search systems are indispensable in large language model (LLM) serving, search engines, and recommender systems, where minimizing online search latency is essential. Among various algorithms, graph-based vector search (GVS) is particularly popular due to its high search performance and quality. To efficiently serve low-latency GVS, we propose a hardware-algorithm co-design solution including Falcon, a GVS accelerator, and Delayed-Synchronization Traversal (DST), an accelerator-optimized graph traversal algorithm. Falcon implements high-performance GVS operators and reduces memory accesses with an on-chip Bloom filter to track search states. DST improves search performance and quality by relaxing the graph traversal order to maximize accelerator utilization. Evaluation across various graphs and datasets shows that our Falcon prototype on FPGAs, coupled with DST, achieves up to 4.3$ imes$ and 19.5$ imes$ speedups in latency and up to 8.0$ imes$ and 26.9$ imes$ improvements in energy efficiency over CPU and GPU-based GVS systems. The remarkable efficiency of Falcon and DST demonstrates their potential to become the standard solutions for future GVS acceleration.

Problem

• Reducing latency in graph-based vector search systems
• Overcoming hardware limitations for parallel graph traversals
• Co-designing hardware and algorithm for efficient GVS acceleration

Innovation

• Hardware-accelerated GVS with efficient operators
• Delayed-Synchronization Traversal for parallel graph search
• On-chip Bloom filter reduces memory access