Umlaut information

2025-03-24

AI Summary

This work investigates the error exponent characterization of channel coding at zero-rate. Confronting the challenge that classical sphere-packing bounds fail to yield tight single-letter expressions in this regime, we introduce “umlaut information”—a novel single-letter information measure—and prove its additivity over parallel channels, thereby overcoming the fundamental non-additivity limitation of lautum information. Umlaut information admits two operational interpretations: it precisely characterizes the optimal error exponent for both no-signaling-assisted coding and list decoding under the large-list limit. Leveraging information-theoretic analysis, asymptotic derivations of the sphere-packing bound, and large-list-limit techniques for list decoding, we derive a tight single-letter formula and rigorously establish its additivity and optimality. Our results unify the performance limits of two distinct zero-rate coding paradigms.

Technology Category

• Machine Learning: Information Theory
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Game Theory and Economic Paradigms: Imperfect Information

Application Category

• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Security and Privacy: Large-scale security measurements
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

The sphere-packing bound quantifies the error exponent for noisy channel coding for rates above a critical value. Here, we study the zero-rate limit of the sphere-packing bound and show that it has an intriguing single-letter form, which we call the umlaut information of the channel, inspired by the lautum information introduced by Palomar and Verdú. Unlike the latter quantity, we show that the umlaut information is additive for parallel uses of channels. We show that it has a twofold operational interpretation: as the zero-rate error exponent of non-signalling-assisted coding on the one hand, and as the zero-rate error exponent of list decoding in the large list limit on the other.

Problem

• Studies zero-rate limit of sphere-packing bound
• Introduces umlaut information for channel analysis
• Explores operational interpretations in coding scenarios

Innovation

• Umlaut information for zero-rate error exponent
• Additive for parallel channel uses
• Operational in non-signalling and list decoding