Cross-Domain Lossy Compression via Constrained Minimum Entropy Coupling

2026-05-10

AI Summary

This work addresses the challenge of cross-domain lossy compression under source-domain degradation, where rate constraints, classification performance, and alignment with the target-domain distribution must be jointly optimized. To this end, the paper proposes a Minimum Entropy Coupling (MEC) framework incorporating explicit rate and classification constraints. Instead of conventional distortion metrics, the method maximizes the coupling strength between the source and its reconstruction, yielding a deterministic coupling model. Theoretically, it is shown that intermediate representations can be removed without loss, establishing equivalence to deterministic coupling. Closed-form solutions are derived for Bernoulli sources, both with and without classification constraints. Combining information-theoretic optimization with neural restoration networks, experiments on MNIST super-resolution and SVHN denoising demonstrate that increasing the bit rate simultaneously improves classification accuracy and produces more discriminative reconstructions.

Abstract

This paper studies cross-domain lossy compression through the lens of minimum entropy coupling (MEC) with rate and classification constraints. In this setting, an encoder observes samples from a degraded source domain, while the decoder is required to generate outputs following a prescribed target distribution and to preserve information relevant to a downstream classification task. Motivated by logarithmic-loss distortion, we adopt an information-based objective that maximizes the coupling strength between the source and reconstruction, rather than minimizing a sample-wise distortion. Under common randomness, we formulate a rate-constrained MEC problem (MEC-B) and show that the intermediate representation can be removed without loss of optimality, yielding an equivalent deterministic coupling formulation. For Bernoulli sources, closed-form expressions are derived with and without classification constraints. In addition, we implement a neural restoration framework using quantization, entropy modeling, distribution matching, and classification regularization. Experiments on MNIST super-resolution and SVHN denoising show that increasing the available rate improves classification accuracy and yields more informative reconstructions.

Problem

• cross-domain lossy compression
• minimum entropy coupling
• classification constraints
• rate constraint
• distribution matching

Innovation

• minimum entropy coupling
• cross-domain compression
• information-theoretic objective
• classification-aware reconstruction
• deterministic coupling