Channel Simulation and Distributed Compression with Ensemble Rejection Sampling

2025-10-06

AI Summary

This work addresses two fundamental problems—channel simulation and distributed matching—by proposing a novel encoding and matching framework based on Ensemble Rejection Sampling (ERS). For channel simulation, we design an ERS-based encoding scheme that asymptotically achieves the optimal rate over continuous alphabets. For distributed matching, we establish, for the first time within the rejection sampling paradigm, a Distributed Matching Lemma that attains matching probabilities approaching those of the Poisson Matching Lemma (PML), thereby closing a theoretical gap in distributed rejection sampling. Our method integrates ERS, standard rejection sampling, and distributed coding techniques. Empirical evaluation on synthetic Gaussian sources and the MNIST dataset demonstrates that our approach significantly outperforms existing methods in distributed compression tasks, achieving both theoretical optimality and practical efficiency.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization
• Machine Learning: Ensemble Methods

Application Category

• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems

Abstract

We study channel simulation and distributed matching, two fundamental problems with several applications to machine learning, using a recently introduced generalization of the standard rejection sampling (RS) algorithm known as Ensemble Rejection Sampling (ERS). For channel simulation, we propose a new coding scheme based on ERS that achieves a near-optimal coding rate. In this process, we demonstrate that standard RS can also achieve a near-optimal coding rate and generalize the result of Braverman and Garg (2014) to the continuous alphabet setting. Next, as our main contribution, we present a distributed matching lemma for ERS, which serves as the rejection sampling counterpart to the Poisson Matching Lemma (PML) introduced by Li and Anantharam (2021). Our result also generalizes a recent work on importance matching lemma (Phan et al, 2024) and, to our knowledge, is the first result on distributed matching in the family of rejection sampling schemes where the matching probability is close to PML. We demonstrate the practical significance of our approach over prior works by applying it to distributed compression. The effectiveness of our proposed scheme is validated through experiments involving synthetic Gaussian sources and distributed image compression using the MNIST dataset.

Problem

• Achieving near-optimal coding rates for channel simulation problems
• Developing distributed matching lemma for ensemble rejection sampling schemes
• Applying improved compression methods to synthetic and image data

Innovation

• Ensemble Rejection Sampling for channel simulation
• Distributed matching lemma for rejection sampling schemes
• Applied to distributed compression with MNIST validation