The Performance of Compression-Based Denoisers

2025-12-16

AI Summary

This paper addresses universal denoising over discrete memoryless channels (DMCs), overcoming the limitation of conventional compression-based denoising—restricted to additive-noise channels—by proposing the first compression-based denoising framework applicable to arbitrary DMCs. Methodologically, it aligns the distortion metric with the channel’s conditional distribution and integrates universal lossless or lossy compressors for robust reconstruction. Theoretically, it establishes the first tight bound on the deviation of the empirical joint distribution from Markovity, fully characterizing the asymptotically optimal distortion and rigorously quantifying performance under both mean-squared error and Hamming distortion. Its core contribution is revealing a fundamental connection between compression-based denoising and the indirect rate-distortion function, thereby unifying and generalizing prior results. This work provides both an information-theoretic foundation and a practical algorithmic paradigm for data recovery over non-additive channels.

Technology Category

• Machine Learning: Information Theory
• Data Mining & Knowledge Management: Data Compression
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

We consider a denoiser that reconstructs a stationary ergodic source by lossily compressing samples of the source observed through a memoryless noisy channel. Prior work on compression-based denoising has been limited to additive noise channels. We extend this framework to general discrete memoryless channels by deliberately choosing the distortion measure for the lossy compressor to match the channel conditional distribution. By bounding the deviation of the empirical joint distribution of the source, observation, and denoiser outputs from satisfying a Markov property, we give an exact characterization of the loss achieved by such a denoiser. Consequences of these results are explicitly demonstrated in special cases, including for MSE and Hamming loss. A comparison is made to an indirect rate-distortion perspective on the problem.

Problem

• Extends compression-based denoising to general discrete memoryless channels
• Characterizes denoising loss by bounding empirical distribution deviations
• Demonstrates results for specific distortion measures like MSE and Hamming

Innovation

• Extends compression-based denoising to general discrete memoryless channels
• Chooses distortion measure to match channel conditional distribution
• Bounds deviation from Markov property to characterize loss