MemANNS: Enhancing Billion-Scale ANNS Efficiency with Practical PIM Hardware

2024-10-31

arXiv.org

AI Summary

To address the dual challenges of memory-bandwidth bottlenecks on CPUs and suboptimal resource utilization on GPUs—both leading to low throughput and energy efficiency in billion-scale approximate nearest neighbor search (ANNS)—this paper re-engineers the IVFPQ algorithm for UPMEM’s Processing-in-Memory (PIM) architecture. We introduce three key innovations: (1) an architecture-aware data layout and query scheduling strategy; (2) a PIM-optimized multi-threaded cooperative execution mechanism; and (3) a compact IVFPQ encoding method leveraging vector co-occurrence patterns to minimize memory access overhead. Evaluated on real UPMEM hardware, our implementation achieves 4.3× higher throughput than CPU-based Faiss, matches GPU-Faiss performance, and improves energy efficiency by 2.3× (QPS/Watt). This work presents the first empirical validation of PIM’s advantages for large-scale vector retrieval—demonstrating both high throughput and low power consumption.

Technology Category

• Machine Learning: Hardware-aware ML
• Search and Optimization: Distributed Search
• Data Mining & Knowledge Management: Intelligent Query Processing

Application Category

• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Graph Algorithms and Modeling for the Web: Querying, indexing, and retrieval in Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

In numerous production environments, Approximate Nearest Neighbor Search (ANNS) plays an indispensable role, particularly when dealing with massive datasets that can contain billions of entries. The necessity for rapid response times in these applications makes the efficiency of ANNS algorithms crucial. However, traditional ANNS approaches encounter substantial challenges at the billion-scale level. CPU-based methods are hindered by the limitations of memory bandwidth, while GPU-based methods struggle with memory capacity and resource utilization efficiency. This paper introduces MemANNS, an innovative framework that utilizes UPMEM PIM architecture to address the memory bottlenecks in ANNS algorithms at scale. We concentrate on optimizing a well-known ANNS algorithm, IVFPQ, for PIM hardware through several techniques. First, we introduce an architecture-aware strategy for data placement and query scheduling that ensures an even distribution of workload across PIM chips, thereby maximizing the use of aggregated memory bandwidth. Additionally, we have developed an efficient thread scheduling mechanism that capitalizes on PIM's multi-threading capabilities and enhances memory management to boost cache efficiency. Moreover, we have recognized that real-world datasets often feature vectors with frequently co-occurring items. To address this, we propose a novel encoding method for IVFPQ that minimizes memory accesses during query processing. Our comprehensive evaluation using actual PIM hardware and real-world datasets at the billion-scale, show that MemANNS offers a significant 4.3x increase in QPS over CPU-based Faiss, and it matches the performance of GPU-based Faiss implementations. Additionally, MemANNS improves energy efficiency, with a 2.3x enhancement in QPS/Watt compared to GPU solutions.

Problem

• Accelerating billion-scale ANNS with PIM architecture
• Overcoming CPU memory bandwidth and GPU underutilization bottlenecks
• Improving performance and energy efficiency for large datasets

Innovation

• Leveraging PIM architecture for billion-scale ANNS
• Dynamic resource management for optimal PIM utilization
• Co-occurrence optimized encoding reducing redundant computations