LotteryCodec: Searching the Implicit Representation in a Random Network for Low-Complexity Image Compression

2025-07-01

AI Summary

Image compression typically relies on large-scale, weight-intensive training, hindering deployment on resource-constrained devices. Method: This paper proposes the “Lottery Encoder” hypothesis—that a randomly initialized over-parameterized network contains untrained implicit subnetworks capable of efficient single-image compression. We search for such substructures via binary masks over the random network, optimize mask generation using a backtracking modulation mechanism, and enable adaptive control of mask sparsity to regulate decoding complexity. Contribution/Results: We present the first empirical validation that untrained subnetworks can effectively perform single-image overfitting-based compression. Our framework establishes a lightweight, shared-random-network codec architecture that requires no training. Experiments demonstrate state-of-the-art rate-distortion performance surpassing VTM, while enabling adaptive decoding complexity—offering a flexible, training-free compression solution tailored for heterogeneous hardware.

Technology Category

• Machine Learning: Learning on the Edge & Model Compression
• Computer Vision: Learning & Optimization for CV
• Search and Optimization: Learning to Search

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs

Abstract

We introduce and validate the lottery codec hypothesis, which states that untrained subnetworks within randomly initialized networks can serve as synthesis networks for overfitted image compression, achieving rate-distortion (RD) performance comparable to trained networks. This hypothesis leads to a new paradigm for image compression by encoding image statistics into the network substructure. Building on this hypothesis, we propose LotteryCodec, which overfits a binary mask to an individual image, leveraging an over-parameterized and randomly initialized network shared by the encoder and the decoder. To address over-parameterization challenges and streamline subnetwork search, we develop a rewind modulation mechanism that improves the RD performance. LotteryCodec outperforms VTM and sets a new state-of-the-art in single-image compression. LotteryCodec also enables adaptive decoding complexity through adjustable mask ratios, offering flexible compression solutions for diverse device constraints and application requirements.

Problem

• Untrained subnetworks for image compression
• Over-parameterized network challenges in compression
• Adaptive decoding complexity for diverse devices

Innovation

• Untrained subnetworks enable low-complexity image compression
• Binary mask overfits images in shared random network
• Rewind modulation enhances rate-distortion performance