🤖 AI Summary
This work addresses a critical yet previously unformalized issue in model-driven lossless compression: decoding failures caused by inconsistencies between the probability predictions of the encoder and decoder. To resolve this prediction mismatch problem, the authors propose PMATIC, a model-agnostic fault-tolerant encoding algorithm that introduces a bounded prediction bias tolerance mechanism. This approach maintains theoretical correctness while substantially reducing computational and compression overhead. PMATIC is designed as a drop-in replacement for conventional arithmetic coders and is compatible with any probabilistic prediction model, including deep neural networks. Empirical evaluations on textual data demonstrate that PMATIC achieves superior compression ratios compared to state-of-the-art tools, even in the presence of non-negligible prediction discrepancies.
📝 Abstract
It is well-known in the field of lossless data compression that probabilistic next-symbol prediction can be used to compress sequences of symbols. Deep neural networks are able to capture rich dependencies in data, offering a powerful means of estimating these probabilities and hence an avenue towards more effective compression algorithms. However, both compressor and decompressor must have exactly matching predictions; even small non-deterministic differences (which often happen with learned models due to hardware, software, or computation order) can lead to cascading decoding failures. In this paper, we formalize the problem of prediction mismatch in model-driven compression, and introduce Probability Matching Interval Coding (PMATIC), a model-agnostic algorithm that tolerates bounded prediction mismatch with low overhead. PMATIC works with the predicted probabilities, making it compatible as a drop-in replacement for the arithmetic encoder in model-driven compression tools. We show theoretical correctness and performance bounds for PMATIC, and validate these results on text data. These results confirm that, when paired an advanced prediction model, PMATIC is robust to prediction mismatch while achieving compression rates that out-perform standard modern compression tools.