Lossless data compression at pragmatic rates

2025-01-17

AI Summary

This work addresses the fundamental lossless compression rate bound for short blocklengths under stringent reliability requirements, focusing on i.i.d. sources subject to exponentially small excess-rate probability constraints. Methodologically, it integrates large deviations theory, Gaussian approximation, prefix/non-prefix code analysis, and error exponent inversion—bypassing conventional normal approximations and classical error exponent bounds. The key contribution is the first non-asymptotic, exact characterization of the optimal compression rate with explicit constants, revealing that, in the short-block regime, the optimal rate deviates significantly from entropy and is governed by the inverse error exponent function. The derived bound achieves superior accuracy in the low excess-rate region and provides a verifiable, practical rate design criterion for ultra-low-latency, high-reliability compression systems—such as edge communication and real-time sensing applications.

Technology Category

• Machine Learning: Information Theory
• Data Mining & Knowledge Management: Data Compression
• Search and Optimization: Non-convex Optimization

Application Category

• Security and Privacy: Large-scale security measurements
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

The problem of variable-rate lossless data compression is considered, for codes with and without prefix constraints. Sharp bounds are derived for the best achievable compression rate of memoryless sources, when the excess-rate probability is required to be exponentially small in the blocklength. Accurate nonasymptotic expansions with explicit constants are obtained for the optimal rate, using tools from large deviations and Gaussian approximation. Examples are shown indicating that, in the small excess-rate-probability regime, the approximation to the fundamental limit of the compression rate suggested by these bounds is significantly more accurate than the approximations provided by either normal approximation or error exponents. The new bounds reinforce the crucial operational conclusion that, in applications where the blocklength is relatively short and where stringent guarantees are required on the rate, the best achievable rate is no longer close to the entropy. Rather, it is an appropriate, more pragmatic rate, determined via the inverse error exponent function and the blocklength.

Problem

• Data Compression
• Information Integrity
• Size Limitation

Innovation

• Data Compression
• Accuracy Improvement
• Entropy Calculation超越