Unlocking Fractional Moments in Delphic Set Streams

2026-08-13

AI Summary

This work addresses the problem of efficiently estimating non-integer frequency moments $F_k$ for $k \in (0,1)$ and Bernstein-type statistics in the Delphic set-stream model. It introduces the first single-pass streaming algorithm that treats the distinct count of substreams as an analytic function of the sampling rate via multiscale sampling. Leveraging a connection to Laplace-type integrals, the algorithm reconstructs the target statistics using numerical integration and function approximation, thereby establishing a unified estimation framework. Under practical settings where $\tau = \mathrm{polylog}(|\Omega|, m)$, the method achieves space and per-update time complexities of $\mathrm{poly}(\log|\Omega|, \log m, \varepsilon^{-1}, \log(1/\delta))$. In the general case, the complexity is polynomial in $\tau$ and $\varepsilon^{-1}$, logarithmic in $\delta^{-1}$, and reveals fundamental lower-bound barriers under the unbounded frequency assumption.

Abstract

We consider estimation of non-integer frequency moments $F_k$ and related Bernstein-type statistics in the Delphic set stream model under a bounded-frequency assumption: every universe element appears at most $τ$ times. The main challenge of this model is to keep space low while also keeping update time low, which is not trivial because the sets can be exponential in size compared to their representations. Our core insight is that by sampling the stream at different rates and observing the resulting distinct-counts, we can 'probe' the frequency distribution and numerically integrate these probes to reconstruct a broad class of statistics. Building on that, we crucially observe that the distinct-count of a randomly sampled substream, viewed as a function of the sampling rate, is a single analytic object whose evaluations determine a broad class of statistics via a complementary Laplace-type integral. Algorithmically we exploit this by: 1. estimating those evaluations using only standard $F_0$ (distinct-count) algorithms on sampled substreams and 2. recovering target statistics by controlled numerical integration on a judiciously chosen grid. For $F_k$ with $k\in (0,1)$ we obtain the first one-pass streaming algorithms for Delphic set streams whose space and per-set update time are $\mathrm{poly}(\log|Ω|,\log m,\varepsilon^{-1},\log(1/δ))$ in the practically relevant regime $τ=\mathrm{polylog}(|Ω|,m)$; in general the bounds are polynomial in $τ$ and $\varepsilon^{-1}$ and logarithmic in $δ^{-1}$. We also give a complexity-theoretic barrier explaining why lower bounds for removing the bounded-frequency assumption appear difficult: ruling out polylogarithmic algorithms for unrestricted Delphic $F_k$ would imply a linear-space threshold-counting separation.

Problem

• fractional frequency moments
• Delphic set streams
• bounded-frequency assumption
• streaming algorithms
• non-integer moments

Innovation

• fractional frequency moments
• Delphic set streams
• distinct-count sampling
• numerical integration
• streaming algorithms