Cost-Effective, Low Latency Vector Search with Azure Cosmos DB

2025-05-09

AI Summary

Cloud-native transactional databases (e.g., Azure Cosmos DB) lack efficient vector search capabilities. Method: This work pioneers the native integration of DiskANN—a high-performance disk-based indexing library—into a NoSQL engine, enabling unified storage and strong consistency between vector indexes and primary data. We propose a distributed index synchronization protocol, adaptive sharding, and memory-mapped file optimizations. Contribution/Results: On 10M vectors, the system achieves <20 ms P95 latency, high recall, and stable update throughput. Compared to Zilliz and Pinecone Serverless, query cost is reduced by 15× and 41×, respectively, while supporting elastic scaling to 1B–10B vectors. Crucially, it retains cloud database advantages—including high availability, durability, and horizontal scalability—without requiring a dedicated vector database, establishing a new production-grade paradigm for semantic search that is low-cost, low-latency, and strongly consistent.

Technology Category

• Search and Optimization: Distributed Search
• Data Mining & Knowledge Management: Scalability, Parallel & Distributed Systems
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.

Application Category

• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Querying, indexing, and retrieval in Web-related graphs

Abstract

Vector indexing enables semantic search over diverse corpora and has become an important interface to databases for both users and AI agents. Efficient vector search requires deep optimizations in database systems. This has motivated a new class of specialized vector databases that optimize for vector search quality and cost. Instead, we argue that a scalable, high-performance, and cost-efficient vector search system can be built inside a cloud-native operational database like Azure Cosmos DB while leveraging the benefits of a distributed database such as high availability, durability, and scale. We do this by deeply integrating DiskANN, a state-of-the-art vector indexing library, inside Azure Cosmos DB NoSQL. This system uses a single vector index per partition stored in existing index trees, and kept in sync with underlying data. It supports<20ms query latency over an index spanning 10 million of vectors, has stable recall over updates, and offers nearly 15x and 41x lower query cost compared to Zilliz and Pinecone serverless enterprise products. It also scales out to billions of vectors via automatic partitioning. This convergent design presents a point in favor of integrating vector indices into operational databases in the context of recent debates on specialized vector databases, and offers a template for vector indexing in other databases.

Problem

• Enabling efficient semantic search via vector indexing in databases
• Reducing query latency and cost in vector search systems
• Integrating vector indices into operational databases for scalability

Innovation

• Integrates DiskANN into Azure Cosmos DB
• Uses single vector index per partition
• Scales to billions via automatic partitioning