On entropy-constrained Gaussian channel capacity via the moment problem

2025-01-23

AI Summary

This paper investigates the capacity limit of the additive white Gaussian noise (AWGN) channel under joint entropy and average power constraints in the low-SNR regime. Addressing realistic input distribution constraints—simultaneous bounds on average power and small differential entropy—we propose a moment-matching-based analytical framework. We first prove that any continuous random variable can be exactly matched up to its first three moments by a discrete distribution whose differential entropy tends to zero; moreover, this third-order moment matching is optimal under entropy constraints. Leveraging moment problem theory, information-theoretic analysis, and distribution approximation, we derive an asymptotically tight expression for channel capacity at low SNR. Our key contribution is establishing that the channel capacity under entropy constraints is fully characterized by the first three moments of the input distribution—breaking the classical paradigm reliant solely on the second moment (power)—thereby providing a fundamental theoretical benchmark for ultra-low-power communication system design.

Technology Category

• Machine Learning: Information Theory
• Constraint Satisfaction and Optimization: Other Foundations of Constraint Satisfaction
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Security and Privacy: Large-scale security measurements
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization

Abstract

We study the capacity of the power-constrained additive Gaussian channel with an entropy constraint at the input. In particular, we characterize this capacity in the low signal-to-noise ratio regime, as a corollary of the following general result on a moment matching problem: we show that for any continuous random variable with finite moments, the largest number of initial moments that can be matched by a discrete random variable of sufficiently small but positive entropy is three.

Problem

• Information Transmission
• Additive Gaussian Noise Channel
• Low Signal-to-Noise Ratio

Innovation

• Low Signal-to-Noise Ratio
• Additive Gaussian Channel
• Information Transmission Capacity