Variable-Length Feedback Codes Over Known and Unknown Channels with Non-Vanishing Error Probabilities

2024-01-30

Information Theory Workshop

AI Summary

This work investigates variable-length feedback (VLF) coding over discrete memoryless channels (DMCs) and additive white Gaussian noise (AWGN) channels with ideal feedback, focusing on the critical regime where the average decoding delay tends to infinity while the error probability remains a non-vanishing constant. We propose a universal VLF coding scheme based on empirical mutual information, integrating an enhanced Yamamoto–Itoh stopping rule and refined type-based analysis. Our main contribution is the first derivation of a second-order achievable bound for non-vanishing error probabilities—significantly tightening the classical bound by Polyanskiy et al. (2011). We establish tight, unified second-order asymptotics for both known and unknown channel settings. These results provide more precise fundamental limits and constructive coding strategies for ultra-low-latency reliable communication.

Technology Category

• Machine Learning: Information Theory
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Security and Privacy: Large-scale security measurements
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

We study variable-length feedback (VLF) codes with noiseless feedback for discrete memoryless channels. We present a novel non-asymptotic bound, which analyzes the average error probability and average decoding time of our modified Yamamoto-Itoh scheme. We then optimize the parameters of our code in the asymptotic regime where the average error probability $epsilon$ remains a constant as the average decoding time $N$ approaches infinity. Our second-order achievability bound refines Polyanskiy et al.'s (2011) achievability bound. We also universal-ize our code by employing the empirical mutual information in our decoding metric and derive a second-order achievability bound for universal VLF codes. The proof of our result for universal VLF codes uses a refined version of the method of types and an asymptotic expansion from the nonlinear renewal theory literature.

Problem

• Analyze VLF codes for discrete memoryless channels.
• Optimize code parameters for constant error probability.
• Extend results to additive white Gaussian noise channels.

Innovation

• Variable-length feedback codes
• Non-asymptotic error bound
• Universal decoding metric optimization