Optimal Static Dictionary with Worst-Case Constant Query Time

2024-12-14

arXiv.org

AI Summary

This work addresses the long-standing trade-off between space efficiency and deterministic query performance in static dictionaries. We propose a novel data structure achieving near-information-theoretic optimal space—namely, OPT + n^ε bits—and worst-case O(1) query time. Our construction integrates hierarchical hashing, combinatorial design, precomputed lookup tables, and information-theoretic encoding, augmented by randomized construction techniques. For the full universe [U] and value domain [σ], it is the first to simultaneously achieve OPT + polylog n bits of space and strictly constant-time queries. In contrast to prior approaches—such as Patrascu (2008) and Yu (2020)—which only guarantee expected constant-time queries or incur larger space overhead, our result breaks the conventional impossibility barrier between deterministic low-latency access and near-optimal space usage. This establishes a new theoretical and practical benchmark for static dictionary design.

Technology Category

• Search and Optimization: Learning to Search
• Data Mining & Knowledge Management: Data Compression
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Security and Privacy: Applications of cryptography

Abstract

In this paper, we design a new succinct static dictionary with worst-case constant query time. A dictionary data structure stores a set of key-value pairs with distinct keys in $[U]$ and values in $[sigma]$, such that given a query $xin [U]$, it quickly returns if $x$ is one of the input keys, and if so, also returns its associated value. The textbook solution to dictionaries is hash tables. On the other hand, the (information-theoretical) optimal space to encode such a set of key-value pairs is only $ ext{OPT} := loginom{U}{n}+nlog sigma$. We construct a dictionary that uses $ ext{OPT} + n^{epsilon}$ bits of space, and answers queries in constant time in worst case. Previously, constant-time dictionaries are only known with $ ext{OPT} + n/ ext{poly}log n$ space [Pv{a}trac{s}cu 2008], or with $ ext{OPT}+n^{epsilon}$ space but expected constant query time [Yu 2020]. We emphasize that most of the extra $n^{epsilon}$ bits are used to store a lookup table that does not depend on the input, and random bits for hash functions. The"main"data structure only occupies $ ext{OPT}+ ext{poly}log n$ bits.

Problem

• Design succinct static dictionary with worst-case constant query time
• Achieve optimal space usage with minimal extra bits
• Improve upon previous space-time tradeoffs in dictionary structures

Innovation

• Succinct static dictionary with constant query time
• Optimal space usage with minimal extra bits
• Lookup table and hash functions for efficiency