Approximate Nearest Neighbor Search for Modern AI: A Projection-Augmented Graph Approach

2026-03-01

AI Summary

Existing approximate nearest neighbor (ANN) search methods struggle to simultaneously achieve high query efficiency, fast indexing, low memory consumption, scalability to high dimensions, robustness across varying data scales, and support for online insertions—key requirements in modern AI applications. This work proposes the Projection-Augmented Graph (PAG) framework, which uniquely integrates projection mechanisms into graph-based indexing. By leveraging statistically grounded asymmetric distance comparisons, PAG drastically reduces the need for exact distance computations while jointly optimizing index construction and query processing within a unified architecture. The framework natively supports online insertions and, across six real-world datasets, demonstrates up to 5× higher QPS-recall performance than HNSW, while maintaining efficient indexing, moderate memory overhead, and consistent robustness in both high-dimensional and large-scale settings.

Technology Category

• Search and Optimization: Distributed Search
• Machine Learning: Graph-based Machine Learning
• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community

Application Category

• Graph Algorithms and Modeling for the Web: Querying, indexing, and retrieval in Web-related graphs
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Approximate Nearest Neighbor Search (ANNS) is fundamental to modern AI applications. Most existing solutions optimize query efficiency but fail to align with the practical requirements of modern workloads. In this paper, we outline six critical demands of modern AI applications: high query efficiency, fast indexing, low memory footprint, scalability to high dimensionality, robustness across varying retrieval sizes, and support for online insertions. To satisfy all these demands, we introduce Projection-Augmented Graph (PAG), a new ANNS framework that integrates projection techniques into a graph index. PAG reduces unnecessary exact distance computations through asymmetric comparisons between exact and approximate distances as guided by projection-based statistical tests. Three key components are designed and unified to the graph index to optimize indexing and searching. Experiments on six modern datasets demonstrate that PAG consistently achieves superior query per second (QPS)-recall performance -- up to 5x faster than HNSW -- while offering fast indexing speed and moderate memory footprint. PAG remains robust as dimensionality and retrieval size increase and naturally supports online insertions.

Problem

• Approximate Nearest Neighbor Search
• modern AI applications
• query efficiency
• high dimensionality
• online insertions

Innovation

• Approximate Nearest Neighbor Search
• Projection-Augmented Graph
• Asymmetric Distance Comparison
• Online Insertion
• High-Dimensional Scalability