Seq2Seq2Seq: Lossless Data Compression via Discrete Latent Transformers and Reinforcement Learning

2026-02-12

AI Summary

This work proposes a novel lossless compression method based on the T5 architecture that effectively leverages structural redundancy in complex data while preserving the discrete symbolic nature of the original input. Unlike conventional compression techniques, which struggle to exploit such redundancy, and existing deep learning approaches that rely on continuous vector representations—thereby disrupting the discrete token structure—our method integrates discrete latent representations with off-policy reinforcement learning. Trained end-to-end, it directly optimizes the length of discrete symbol sequences without requiring external knowledge. By maintaining the integrity of the original token structure and semantics, the approach achieves superior compression ratios and enhanced generalization across diverse data types, significantly outperforming traditional compression algorithms.

Technology Category

• Data Mining & Knowledge Management: Data Compression
• Machine Learning: Learning on the Edge & Model Compression
• Search and Optimization: Learning to Search

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies

Abstract

Efficient lossless compression is essential for minimizing storage costs and transmission overhead while preserving data integrity. Traditional compression techniques, such as dictionary-based and statistical methods, often struggle to optimally exploit the structure and redundancy in complex data formats. Recent advancements in deep learning have opened new avenues for compression; however, many existing approaches depend on dense vector representations that obscure the underlying token structure. To address these limitations, we propose a novel lossless compression method that leverages Reinforcement Learning applied to a T5 language model architecture. This approach enables the compression of data into sequences of tokens rather than traditional vector representations. Unlike auto-encoders, which typically encode information into continuous latent spaces, our method preserves the token-based structure, aligning more closely with the original data format. This preservation allows for higher compression ratios while maintaining semantic integrity. By training the model using an off-policy Reinforcement Learning algorithm, we optimize sequence length to minimize redundancy and enhance compression efficiency. Our method introduces an efficient and adaptive data compression system built upon advanced Reinforcement Learning techniques, functioning independently of external grammatical or world knowledge. This approach shows significant improvements in compression ratios compared to conventional methods. By leveraging the latent information within language models, our system effectively compresses data without requiring explicit content understanding, paving the way for more robust and practical compression solutions across various applications.

Problem

• lossless compression
• data redundancy
• token structure
• discrete representation
• compression efficiency

Innovation

• Discrete Latent Transformers
• Reinforcement Learning
• Lossless Compression
• Token-based Representation
• T5 Language Model