Channel-Aware Optimal Transport: A Theoretical Framework for Generative Communication

2024-12-26

AI Summary

This paper investigates the channel-aware optimal transmission problem: transmitting an i.i.d. source sequence over a memoryless channel—without shared randomness—to synthesize an output sequence with a prescribed marginal distribution, while minimizing end-to-end distortion. We propose a hybrid analog-digital coding scheme that breaks from the classical source-channel separation architecture, enabling the first joint optimization of generative fidelity and reliable digital communication. Our theoretical analysis, grounded in optimal transport theory and information theory, derives asymptotic performance bounds. Results show that source-channel separation is asymptotically optimal only when unlimited common randomness is available; in the absence of shared randomness, our hybrid scheme substantially outperforms both separated and uncoded transmission, achieving a significant improvement in the distortion-rate trade-off.

Technology Category

• Search and Optimization: Mixed Discrete/Continuous Search
• Constraint Satisfaction and Optimization: Mixed Discrete/Continuous Optimization
• Machine Learning: Information Theory

Application Category

• Security and Privacy: Large-scale security measurements
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Responsible Web: Measurement, analysis, and circumvention of Web censorship

Abstract

Optimal transport has numerous applications, particularly in machine learning tasks involving generative models. In practice, the transportation process often encounters an information bottleneck, typically arising from the conversion of a communication channel into a rate-limited bit pipeline using error correction codes. While this conversion enables a channel-oblivious approach to optimal transport, it fails to fully exploit the available degrees of freedom. Motivated by the emerging paradigm of generative communication, this paper examines the problem of channel-aware optimal transport, where a block of i.i.d. random variables is transmitted through a memoryless channel to generate another block of i.i.d. random variables with a prescribed marginal distribution such that the end-to-end distortion is minimized. With unlimited common randomness available to the encoder and decoder, the source-channel separation architecture is shown to be asymptotically optimal as the blocklength approaches infinity. On the other hand, in the absence of common randomness, the source-channel separation architecture is generally suboptimal. For this scenario, a hybrid coding scheme is proposed, which partially retains the generative capabilities of the given channel while enabling reliable transmission of digital information. It is demonstrated that the proposed hybrid coding scheme can outperform both separation-based and uncoded schemes.

Problem

• Optimal Transport
• Mixed Encoding Strategy
• Minimum Distortion

Innovation

• Hybrid Coding
• Optimal Transmission
• Randomness Generation