Optimal Non-Oblivious Open Addressing

2025-03-17

AI Summary

Conventional wisdom in open-addressing hash tables posits an inherent space–time trade-off: high load factors inevitably degrade operation latency. Method: The paper introduces a *non-blind* open-addressing hash table that employs dynamic rehashing, low-independence hash functions (requiring only O(1) pairwise-independent O(1)-wise independent hash functions), a non-blind probing strategy, and constructive probabilistic analysis. Contribution/Results: This design achieves high-probability constant-time (O(1)) and expected O(1) worst-case time complexity for insertions, deletions, and lookups—even at full capacity (load factor 1.0, i.e., zero redundant slots). It is the first construction to break the long-standing Ω(log log ε⁻¹) lower bound on probe complexity for fully loaded tables. By simultaneously attaining asymptotic space optimality and strong time efficiency, the scheme establishes a new theoretical and practical paradigm for compact hash structures.

Technology Category

• Search and Optimization: Non-convex Optimization
• Data Mining & Knowledge Management: Data Compression
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Security and Privacy: Data transparency and provenance
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

A hash table is said to be open-addressed (or non-obliviously open-addressed) if it stores elements (and free slots) in an array with no additional metadata. Intuitively, open-addressed hash tables must incur a space-time tradeoff: The higher the load factor at which the hash table operates, the longer insertions/deletions/queries should take. In this paper, we show that no such tradeoff exists: It is possible to construct an open-addressed hash table that supports constant-time operations even when the hash table is entirely full. In fact, it is even possible to construct a version of this data structure that: (1) is dynamically resized so that the number of slots in memory that it uses, at any given moment, is the same as the number of elements it contains; (2) supports $O(1)$-time operations, not just in expectation, but with high probability; and (3) requires external access to just $O(1)$ hash functions that are each just $O(1)$-wise independent. Our results complement a recent lower bound by Bender, Kuszmaul, and Zhou showing that oblivious open-addressed hash tables must incur $Omega(log log varepsilon^{-1})$-time operations. The hash tables in this paper are non-oblivious, which is why they are able to bypass the previous lower bound.

Problem

• Eliminates space-time tradeoff in open-addressed hash tables.
• Supports constant-time operations at full capacity.
• Dynamically resizes with minimal external hash function access.

Innovation

• Open-addressed hash tables with constant-time operations
• Dynamic resizing matching element count to slots
• High probability O(1)-time operations with minimal hash functions