On the Computability of Finding Capacity-Achieving Codes

2025-11-03

AI Summary

This paper addresses the **algorithmic constructibility problem** of capacity-approaching codes over discrete memoryless channels: given a computable channel transition probability, a target rate below channel capacity, and an error tolerance, does there exist a Turing machine that outputs a block code satisfying the prescribed block-error probability constraint? Innovatively grounded in **computability theory**, the paper formally defines this problem for the first time and develops a systematic search framework based on μ-recursive functions. By integrating Shannon’s coding theorem with Kleene’s normal form theorem, it proves that the construction problem is **Turing-decidable** when all parameters are computable real numbers. This result establishes the algorithmic existence of capacity-approaching codes and introduces a novel paradigm bridging information theory and computability theory.

Technology Category

• Machine Learning: Information Theory
• Search and Optimization: Mixed Discrete/Continuous Search
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics

Abstract

This work studies the problem of constructing capacity-achieving codes from an algorithmic perspective. Specifically, we prove that there exists a Turing machine which, given a discrete memoryless channel $p_{Y|X}$, a target rate $R$ less than the channel capacity $C(p_{Y|X})$, and an error tolerance $ε> 0$, outputs a block code $mathcal{C}$ achieving a rate at least $R$ and a maximum block error probability below $ε$. The machine operates in the general case where all transition probabilities of $p_{Y|X}$ are computable real numbers, and the parameters $R$ and $ε$ are rational. The proof builds on Shannon's Channel Coding Theorem and relies on an exhaustive search approach that systematically enumerates all codes of increasing block length until a valid code is found. This construction is formalized using the theory of recursive functions, yielding a $μ$-recursive function $mathrm{FindCode} : mathbb{N}^3 ightharpoonup mathbb{N}$ that takes as input appropriate encodings of $p_{Y|X}$, $R$, and $ε$, and, whenever $R < C(p_{Y|X})$, outputs an encoding of a valid code. By Kleene's Normal Form Theorem, which establishes the computational equivalence between Turing machines and $μ$-recursive functions, we conclude that the problem is solvable by a Turing machine. This result can also be extended to the case where $ε$ is a computable real number, while we further discuss an analogous generalization of our analysis when $R$ is computable as well. We note that the assumptions that the probabilities of $p_{Y|X}$, as well as $ε$ and $R$, are computable real numbers cannot be further weakened, since computable reals constitute the largest subset of $mathbb{R}$ representable by algorithmic means.

Problem

• Algorithmically constructing capacity-achieving error-correcting codes
• Proving Turing machines can find codes for computable channels
• Establishing computability bounds for code synthesis under constraints

Innovation

• Turing machine constructs capacity-achieving codes
• Exhaustive search finds valid codes systematically
• Recursive functions formalize code construction process