History-Independent Concurrent Hash Tables

2025-03-26

AI Summary

This work establishes space lower bounds for history-independent concurrent hash tables—data structures that guarantee linearizability while ensuring memory layouts reveal no information about operation histories. We present the first lock-free, linearizable history-independent concurrent hash table, built upon Robin Hood hashing and LL/SC atomic primitives, with a carefully designed linearization point and rigorous probabilistic analysis. Our main contribution is a tight space lower bound: for two processes, storing exactly two elements per bucket plus a constant number of metadata bits is optimal—no asymptotically smaller solution exists. Specifically, each bucket requires capacity for at most two key-value pairs and two bits of metadata. The algorithm achieves expected amortized step complexity O(1). This is the first work to establish a provably tight space-optimality foundation for history-independent concurrent hash tables, bridging theory and practical design in concurrent data structures.

Technology Category

• Search and Optimization: Distributed Search
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Security and Privacy: Data transparency and provenance
• Economics, Online Markets and Human Computation: The sharing economy

Abstract

A history-independent data structure does not reveal the history of operations applied to it, only its current logical state, even if its internal state is examined. This paper studies history-independent concurrent dictionaries, in particular, hash tables, and establishes inherent bounds on their space requirements. This paper shows that there is a lock-free history-independent concurrent hash table, in which each memory cell stores two elements and two bits, based on Robin Hood hashing. Our implementation is linearizable, and uses the shared memory primitive LL/SC. The expected amortized step complexity of the hash table is $O(c)$, where $c$ is an upper bound on the number of concurrent operations that access the same element, assuming the hash table is not overpopulated. We complement this positive result by showing that even if we have only two concurrent processes, no history-independent concurrent dictionary that supports sets of any size, with wait-free membership queries and obstruction-free insertions and deletions, can store only two elements of the set and a constant number of bits in each memory cell. This holds even if the step complexity of operations on the dictionary is unbounded.

Problem

• Designing history-independent concurrent hash tables
• Establishing space bounds for concurrent dictionaries
• Analyzing trade-offs in lock-free data structures

Innovation

• Lock-free history-independent concurrent hash table
• Uses Robin Hood hashing technique
• Employs LL/SC shared memory primitive