Efficient $varepsilon$-approximate minimum-entropy couplings

2025-09-23

AI Summary

The minimum-entropy coupling problem seeks a joint distribution over $m geq 2$ discrete random variables, each taking values in an $n$-element set, such that its marginal distributions exactly match given input distributions and its Shannon entropy is minimized. A long-standing open question has been whether a polynomial-time approximation scheme (PTAS) exists. This paper provides the first affirmative answer: for constant $m$, it designs the first PTAS—computing, for any $varepsilon > 0$, a coupling with entropy at most $H(mathrm{OPT}) + varepsilon$ in time $n^{O(mathrm{poly}(1/varepsilon) cdot 2^{O(m)})}$. The approach combines techniques from approximation algorithm design with structural analysis of a fixed-dimensional, $m$-dependent exponential configuration space. Crucially, it overcomes the prior restriction to $m = 2$, thereby resolving this fundamental open problem in information theory and probabilistic modeling.

Technology Category

• Constraint Satisfaction and Optimization: Distributed CSP/Optimization
• Machine Learning: Information Theory
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Security and Privacy: Applications of cryptography

Abstract

Given $m ge 2$ discrete probability distributions over $n$ states each, the minimum-entropy coupling is the minimum-entropy joint distribution whose marginals are the same as the input distributions. Computing the minimum-entropy coupling is NP-hard, but there has been significant progress in designing approximation algorithms; prior to this work, the best known polynomial-time algorithms attain guarantees of the form $H(operatorname{ALG}) le H(operatorname{OPT}) + c$, where $c approx 0.53$ for $m=2$, and $c approx 1.22$ for general $m$ [CKQGK '23]. A main open question is whether this task is APX-hard, or whether there exists a polynomial-time approximation scheme (PTAS). In this work, we design an algorithm that produces a coupling with entropy $H(operatorname{ALG}) le H(operatorname{OPT}) + varepsilon$ in running time $n^{O(operatorname{poly}(1/varepsilon) cdot operatorname{exp}(m) )}$: showing a PTAS exists for constant $m$.

Problem

• Computing minimum-entropy couplings for multiple discrete distributions
• Addressing NP-hard nature of exact minimum-entropy coupling computation
• Developing polynomial-time approximation scheme for constant distributions

Innovation

• Polynomial-time approximation scheme for minimum-entropy couplings
• Achieves additive epsilon error in entropy approximation
• Efficient algorithm for constant number of distributions