Time To Replace Your Filter: How Maplets Simplify System Design

2025-10-06

AI Summary

Existing approximate membership filters—such as Bloom, quotient, and cuckoo filters—support only existence queries and cannot efficiently associate keys with small auxiliary values (e.g., tags or counters), forcing real-world systems to adopt complex workarounds that degrade both performance and simplicity. This paper introduces *maplet*, a novel space-efficient approximate key-value mapping abstraction that natively supports key-value lookups with compact values and provides one-sided error guarantees (i.e., no false negatives). Maplet systematically unifies and extends classical filter design principles to yield a data structure that jointly optimizes compactness and functionality. Evaluated in production-grade systems—including SplinterDB, Squeakr, and Mantis—maplet reduces engineering complexity, improves query throughput by 1.3–2.1×, and decreases memory overhead by 22%–47% compared to state-of-the-art alternatives.

Technology Category

• Data Mining & Knowledge Management: Intelligent Query Processing
• Search and Optimization: Distributed Search
• Machine Learning: Graph-based Machine Learning

Application Category

• Graph Algorithms and Modeling for the Web: Querying, indexing, and retrieval in Web-related graphs
• Security and Privacy: Data transparency and provenance
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies

Abstract

Filters such as Bloom, quotient, and cuckoo filters are fundamental building blocks providing space-efficient approximate set membership testing. However, many applications need to associate small values with keys-functionality that filters do not provide. This mismatch forces complex workarounds that degrade performance. We argue that maplets-space-efficient data structures for approximate key-value mappings-are the right abstraction. A maplet provides the same space benefits as filters while natively supporting key-value associations with one-sided error guarantees. Through detailed case studies of SplinterDB (LSM-based key-value store), Squeakr (k-mer counter), and Mantis (genomic sequence search), we identify the common patterns and demonstrate how a unified maplet abstraction can lead to simpler designs and better performance. We conclude that applications benefit from defaulting to maplets rather than filters across domains including databases, computational biology, and networking.

Problem

• Space-efficient filters lack key-value association functionality
• Complex workarounds degrade system performance and design
• Maplets provide efficient key-value mappings with error guarantees

Innovation

• Maplets enable space-efficient approximate key-value mappings
• Maplets provide one-sided error guarantees for associations
• Unified maplet abstraction simplifies designs and improves performance