Modern Minimal Perfect Hashing: A Survey

2025-06-06

AI Summary

This work addresses the long-standing trade-off among space efficiency, construction time, and query latency in modern minimal perfect hash functions (MPHFs). We propose a unified framework integrating hypergraph modeling, randomized construction, hierarchical hashing, and bit-level compression. For the first time in nearly 28 years, our approach achieves space usage within 0.1% of the information-theoretic lower bound (log₂(e) ≈ 1.4427 bits/key). It enables millisecond-scale construction for billion-key sets and nanosecond-scale, single-memory-access queries. We establish a large-scale empirical benchmark spanning up to one billion keys and validate effectiveness across database indexing and bioinformatics applications. All code, experimental infrastructure, and a practical MPHF selection guide are open-sourced. Our work bridges theoretical limits with real-world deployment, providing both rigorous foundations and actionable engineering paradigms for MPHF adoption.

Technology Category

• Data Mining & Knowledge Management: Data Compression
• Search and Optimization: Metareasoning and Metaheuristics
• Machine Learning: Hardware-aware ML

Application Category

• Graph Algorithms and Modeling for the Web: Querying, indexing, and retrieval in Web-related graphs
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Given a set $S$ of $n$ keys, a perfect hash function for $S$ maps the keys in $S$ to the first $m geq n$ integers without collisions. It may return an arbitrary result for any key not in $S$ and is called minimal if $m = n$. The most important parameters are its space consumption, construction time, and query time. Years of research now enable modern perfect hash functions to be extremely fast to query, very space-efficient, and scale to billions of keys. Different approaches give different trade-offs between these aspects. For example, the smallest constructions get within 0.1% of the space lower bound of $log_2(e)$ bits per key. Others are particularly fast to query, requiring only one memory access. Perfect hashing has many applications, for example to avoid collision resolution in static hash tables, and is used in databases, bioinformatics, and stringology. Since the last comprehensive survey in 1997, significant progress has been made. This survey covers the latest developments and provides a starting point for getting familiar with the topic. Additionally, our extensive experimental evaluation can serve as a guide to select a perfect hash function for use in applications.

Problem

• Survey modern minimal perfect hashing techniques
• Compare space, construction, and query time trade-offs
• Guide selection of perfect hash functions for applications

Innovation

• Minimal perfect hashing with zero collisions
• Space-efficient near theoretical lower bound
• Single memory access for fast querying