Conditional Entropies of k-Deletion/Insertion Channels

2024-07-13

arXiv.org

AI Summary

This work investigates extremal conditional entropy problems under $k$-deletion/insertion channels. For binary sequences, we employ combinatorial analysis, run-length encoding, and entropy extremization theory to rigorously characterize the extremal behavior of input and output entropy under 1- and 2-deletion/insertion channels. We prove, for the first time, that in the 1-deletion channel, input entropy is maximized by alternating sequences and minimized by constant sequences—fully confirming Atashpendar et al.’s conjecture on alternating sequences maximizing input entropy; this result extends to the 2-deletion channel. Furthermore, we establish a positive correlation between the balance of run-length distribution and input entropy, and provide a complete characterization of entropy-extremizing sequences for both 1- and 2-deletion/insertion channels. These findings deliver foundational theoretical support for capacity estimation of deletion/insertion channels and the design of robust coding schemes.

Technology Category

• Machine Learning: Information Theory
• Constraint Satisfaction and Optimization: Satisfiability
• Search and Optimization: Combinatorial Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Web Mining and Content Analysis: Models for Web evolution
• Security and Privacy: Large-scale security measurements

Abstract

The channel output entropy of a transmitted sequence is the entropy of the possible channel outputs and similarly the channel input entropy of a received sequence is the entropy of all possible transmitted sequences. The goal of this work is to study these entropy values for the k-deletion, k-insertion channels, where exactly k symbols are deleted, inserted in the transmitted sequence, respectively. If all possible sequences are transmitted with the same probability then studying the input and output entropies is equivalent. For both the 1-deletion and 1-insertion channels, it is proved that among all sequences with a fixed number of runs, the input entropy is minimized for sequences with a skewed distribution of their run lengths and it is maximized for sequences with a balanced distribution of their run lengths. Among our results, we establish a conjecture by Atashpendar et al. which claims that for the 1-deletion channel, the input entropy is maximized by the alternating sequences over all binary sequences. This conjecture is also verified for the 2-deletion channel, where it is proved that constant sequences with a single run minimize the input entropy.

Problem

• Study entropy in k-deletion/insertion channels
• Determine input/output entropy for skewed vs balanced sequences
• Verify conjecture on entropy maximization by alternating sequences

Innovation

• Analyzes entropy in k-deletion/insertion channels
• Proves input entropy minimized for skewed run lengths
• Verifies conjecture for alternating sequences maximizing entropy