MicroNN: An On-device Disk-resident Updatable Vector Database

2025-04-08

AI Summary

To address the challenges of memory constraints, dynamic index updates (insertions/deletions), and hybrid queries (vector approximate nearest neighbor search + structured attribute filtering) on low-resource edge devices, this paper proposes the first lightweight, embeddable, disk-resident, and real-time updatable vector index architecture. Methodologically, it integrates a compact disk-based index design, incremental insertion/deletion mechanisms, multi-condition joint query optimization, and a memory-aware approximate nearest neighbor algorithm. Evaluated on million-scale benchmarks, the system achieves 90% recall using only ~10 MB of RAM, with top-100 retrieval latency under 7 ms. This work is the first to enable high-accuracy, low-latency, and highly dynamic hybrid search on edge devices under extreme memory budgets. It has been successfully deployed across multiple real-world edge scenarios.

Technology Category

• Machine Learning: Learning on the Edge & Model Compression
• Search and Optimization: Distributed Search
• Computer Vision: Image and Video Retrieval

Application Category

• Search and Retrieval-Augmented AI: Web crawling and indexing
• Graph Algorithms and Modeling for the Web: Querying, indexing, and retrieval in Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Nearest neighbour search over dense vector collections has important applications in information retrieval, retrieval augmented generation (RAG), and content ranking. Performing efficient search over large vector collections is a well studied problem with many existing approaches and open source implementations. However, most state-of-the-art systems are generally targeted towards scenarios using large servers with an abundance of memory, static vector collections that are not updatable, and nearest neighbour search in isolation of other search criteria. We present Micro Nearest Neighbour (MicroNN), an embedded nearest-neighbour vector search engine designed for scalable similarity search in low-resource environments. MicroNN addresses the problem of on-device vector search for real-world workloads containing updates and hybrid search queries that combine nearest neighbour search with structured attribute filters. In this scenario, memory is highly constrained and disk-efficient index structures and algorithms are required, as well as support for continuous inserts and deletes. MicroNN is an embeddable library that can scale to large vector collections with minimal resources. MicroNN is used in production and powers a wide range of vector search use-cases on-device. MicroNN takes less than 7 ms to retrieve the top-100 nearest neighbours with 90% recall on publicly available million-scale vector benchmark while using ~10 MB of memory.

Problem

• Efficient on-device vector search in low-resource environments
• Support for real-time updates and hybrid search queries
• Disk-efficient index structures for memory-constrained devices

Innovation

• On-device disk-resident vector database
• Supports updates and hybrid search queries
• Minimal memory usage with high efficiency