SpANNS: Optimizing Approximate Nearest Neighbor Search for Sparse Vectors Using Near Memory Processing

2026-01-06

arXiv.org

AI Summary

This work addresses the challenge of achieving both high scalability and efficiency in approximate nearest neighbor search (ANNS) over sparse vectors on conventional CPU architectures. To this end, we propose SpANNS—the first near-memory computing architecture tailored for sparse ANNS—built upon the CXL Type-2 platform. SpANNS integrates a hybrid inverted index, query parsing, clustering-based filtering, and a compute-enabled DIMM co-processing mechanism to perform index traversal and distance computation efficiently near the data. Evaluated against the state-of-the-art CPU baseline, SpANNS achieves a speedup of 15.2× to 21.6×, substantially enhancing both performance and scalability for sparse vector retrieval.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Search and Optimization: Distributed Search
• Machine Learning: Hardware-aware ML

Application Category

• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Graph Algorithms and Modeling for the Web: Querying, indexing, and retrieval in Web-related graphs
• Social Networks and Social Media: Social mining and social search on the Web

Abstract

Approximate Nearest Neighbor Search (ANNS) is a fundamental operation in vector databases, enabling efficient similarity search in high-dimensional spaces. While dense ANNS has been optimized using specialized hardware accelerators, sparse ANNS remains limited by CPU-based implementations, hindering scalability. This limitation is increasingly critical as hybrid retrieval systems, combining sparse and dense embeddings, become standard in Information Retrieval (IR) pipelines. We propose SpANNS, a near-memory processing architecture for sparse ANNS. SpANNS combines a hybrid inverted index with efficient query management and runtime optimizations. The architecture is built on a CXL Type-2 near-memory platform, where a specialized controller manages query parsing and cluster filtering, while compute-enabled DIMMs perform index traversal and distance computations close to the data. It achieves 15.2x to 21.6x faster execution over the state-of-the-art CPU baselines, offering scalable and efficient solutions for sparse vector search.

Problem

• Approximate Nearest Neighbor Search
• Sparse Vectors
• Near Memory Processing
• Vector Databases
• Information Retrieval

Innovation

• Approximate Nearest Neighbor Search
• Sparse Vectors
• Near Memory Processing
• CXL
• Vector Database