Entropy of Bernoulli Measures Conditioned on Affine Subspaces and a Problem of Ancheta--Massey

2026-08-24

AI Summary

本文解决了Bernoulli源在特定参数下的线性编码最优率问题，通过限制条件下的熵边界方法，并扩展了Ancheta的结果至所有p<1/2的情况。

Abstract

A textbook result in information theory is that linear encoders achieve the entropy for lossless compression of Bernoulli source with parameter $p$. For lossy compression, however, linearity is known to incur strict suboptimality compared to the rate-distortion function. Massey asked whether the optimal rate for linear encoding is achieved simply by compressing a fraction of the bits linearly and losslessly and estimating the rest by zero \cite{Massey1978}. For $p=\frac12$, Ancheta answered this question affirmatively \cite{Ancheta1978}. This note extends Ancheta's result to all $p<\frac12$. The key argument is to bound the entropy of the posterior distribution conditioned on an affine subspace in terms of its marginals. The proof was discovered by GPT-5.6 Sol in an interactive process guided by the author. The purpose of the present note is to communicate a simplified version of this proof and to make connections with the existing literature on coding theory and spin glass theory.

Problem

• linear encoding
• Bernoulli source
• rate-distortion function
• entropy

Innovation

• Bernoulli source
• linear encoding
• affine subspace
• entropy bound
• GPT-5.6