Single-shot lossy compression: mutual information bounds

2026-02-07

AI Summary

This work investigates the minimal expected description length in single-shot lossy compression under three fidelity constraints: guaranteed distortion, conditional excess distortion, and excess distortion. By leveraging information-theoretic analysis and a single-shot extension of rate-distortion theory, the study establishes, for the first time, tight mutual information upper bounds for each constraint and provides equivalent characterizations via their convex surrogates, thereby revealing structural properties of the optimal solutions. The theoretical results demonstrate that, provided the data contains sufficient information content, minimizing mutual information effectively approximates the minimal description length. This finding offers a rigorous theoretical foundation and a practical optimization objective for single-shot lossy compression.

Technology Category

• Machine Learning: Information Theory
• Data Mining & Knowledge Management: Data Compression
• Constraint Satisfaction and Optimization: Other Foundations of Constraint Satisfaction

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Security and Privacy: Large-scale security measurements
• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization

Abstract

For several styles of fidelity constraints -- guaranteed distortion, conditional excess distortion, excess distortion -- we show mutual information upper bounds on the minimum expected description length needed to represent a random variable. Coupled with the corresponding converses, these results attest that as long as the information content in the data is not too low, minimizing the mutual information under an appropriate fidelity constraint serves as a reasonable proxy for the minimum description length of the data. We provide alternative characterizations of all three convex proxies, shedding light on the structure of their solutions.

Problem

• lossy compression
• mutual information
• fidelity constraints
• description length
• distortion

Innovation

• single-shot compression
• mutual information bounds
• lossy compression
• fidelity constraints
• minimum description length