Rényi divergence guarantees for hashing with linear codes

2024-05-07

arXiv.org

AI Summary

This work addresses the problem of extracting uniform random bits from an unknown source with a given Rényi $p$-entropy ($p geq 2$) via linear hashing. We propose a distribution smoothing technique based on random linear codes and analyze the effect of the induced additive noise on the source distribution. This yields the first universal convergence guarantee in Rényi divergence: the expected $p$-divergence between the hash output and the uniform distribution decays exponentially with the code length. Furthermore, we prove that Reed–Muller matrices achieve the intrinsic randomness limit for Bernoulli sources under the $ell_p$ norm—attaining information-theoretically optimal $p$-entropy extraction. This work pioneers the systematic application of Rényi divergence analysis to linear randomness extraction, establishing a new theoretical framework for characterizing the performance of structured hash functions.

Technology Category

• Machine Learning: Information Theory
• Reasoning under Uncertainty: Stochastic Optimization
• Natural Language Processing: Information Extraction

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Security and Privacy: Large-scale security measurements
• Web Mining and Content Analysis: Models for Web evolution

Abstract

We consider the problem of distilling uniform random bits from an unknown source with a given $p$-entropy using linear hashing. As our main result, we estimate the expected $p$-divergence from the uniform distribution over the ensemble of random linear codes for all integer $pge 2$. The proof relies on analyzing how additive noise, determined by a random element of the code from the ensemble, acts on the source distribution. This action leads to the transformation of the source distribution into an approximately uniform one, a process commonly referred to as distribution smoothing. We also show that hashing with Reed-Muller matrices reaches intrinsic randomness of memoryless Bernoulli sources in the $l_p$ sense for all integer $pge 2$.

Problem

• Distilling uniform random bits from unknown p-entropy sources
• Estimating p-divergence for random linear codes ensembles
• Achieving intrinsic randomness for Bernoulli sources via Reed-Muller hashing

Innovation

• Hashing with linear codes for uniformity
• Analyzing additive noise for distribution smoothing
• Reed-Muller matrices ensure intrinsic randomness