Hardness of Frequency-Related Queries on Compressed Strings

2026-07-08

AI Summary

This work addresses the long-standing gap in data structures that simultaneously achieve low space and efficient query time for frequency-related queries—such as rank and symbol occurrence—on grammar-compressed strings. For the first time, it establishes conditional lower bounds for such queries under both grammar-based and LZ78 compression models, revealing deep connections to fundamental conjectures in computational complexity, including Boolean matrix multiplication and the Orthogonal Vectors Hypothesis. Leveraging batched query lower bound techniques, the study shows that supporting rank queries in $O(|G|\log^{O(1)} n)$ space and $\mathrm{polylog}(n)$ time would imply faster algorithms for Boolean matrix multiplication. These results extend to approximate variants and related problems like range distinct count and range mode frequency, thereby characterizing the inherent complexity of frequency queries in compressed indexing.

Abstract

Compressed indexing aims to support fundamental string queries in space proportional to compressed input size. For grammar compression, a length-$n$ string $T \in Σ^n$ represented by a grammar of size $|G|$ can support random access in $O(|G|\log^{O(1)} n)$ space and $O(\log^{O(1)} n)$ time, and the same bounds are known for many other queries, including pattern matching, longest common extension, lexicographic predecessor/successor, the Burrows-Wheeler transform, suffix arrays, and suffix trees. Frequency-related queries remain less understood. These include rank queries, which report the number of occurrences of a symbol $c \in Σ$ in a substring $T(b..e]$, and symbol occurrence queries, which ask whether $c$ occurs in $T(b..e]$. No fully general data structure is known for these queries with $O(|G|\log^{O(1)} n)$ space and $O(\log^{O(1)} n)$ query time. We establish new conditional lower bounds for such problems. First, we show that answering rank and symbol occurrence queries on grammar-compressed texts in polylogarithmic time using an $O(|G|\log^{O(1)} n)$-space structure constructible in $O(|G|\log^{O(1)} n)$ time would imply an $O(n^2\log^{O(1)} n)$-time algorithm for Boolean Matrix Multiplication. The proof uses a more general lower bound for efficiently answering a batch of such queries. Second, we extend the exact lower bounds from straight-line programs to LZ78-compressed strings, a weaker compression model. Third, independently, we show that even additive approximations of rank queries on straight-line grammars would imply faster Boolean Matrix Multiplication algorithms. Finally, assuming the Orthogonal Vectors conjecture, we show that other frequency-related problems, including range distinct counting and range mode frequency, also cannot be efficiently supported in compressed space.

Problem

• compressed indexing
• frequency-related queries
• rank queries
• grammar compression
• conditional lower bounds

Innovation

• compressed indexing
• frequency queries
• conditional lower bounds
• grammar compression
• Boolean Matrix Multiplication