Rethinking Autoregressive Models for Lossless Image Compression via Hierarchical Parallelism and Progressive Adaptation

2025-11-14

AI Summary

Autoregressive (AR) models, while theoretically optimal for lossless image compression, have long been deemed impractical due to prohibitive computational overhead. To address this, we propose HPAC—a hierarchical probabilistic autoregressive compression framework—that enables efficient spatial dependency modeling via hierarchical factorization, content-aware convolutional gating, and spatially aware rate-guided progressive fine-tuning. We further introduce a Cache-then-Select inference mechanism, adaptive focus encoding, and instance-level lightweight fine-tuning using low-rank adapters. Evaluated on natural, satellite, and medical images, HPAC achieves state-of-the-art lossless compression ratios with significantly fewer parameters and competitive encoding speed. Our work is the first to demonstrate that purely autoregressive models can simultaneously attain top-tier compression performance and practical efficiency, thereby reestablishing AR modeling as a viable and foundational paradigm for lossless image compression.

Technology Category

• Machine Learning: Learning on the Edge & Model Compression
• Computer Vision: Learning & Optimization for CV
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

Autoregressive (AR) models, the theoretical performance benchmark for learned lossless image compression, are often dismissed as impractical due to prohibitive computational cost. This work re-thinks this paradigm, introducing a framework built on hierarchical parallelism and progressive adaptation that re-establishes pure autoregression as a top-performing and practical solution. Our approach is embodied in the Hierarchical Parallel Autoregressive ConvNet (HPAC), an ultra-lightweight pre-trained model using a hierarchical factorized structure and content-aware convolutional gating to efficiently capture spatial dependencies. We introduce two key optimizations for practicality: Cache-then-Select Inference (CSI), which accelerates coding by eliminating redundant computations, and Adaptive Focus Coding (AFC), which efficiently extends the framework to high bit-depth images. Building on this efficient foundation, our progressive adaptation strategy is realized by Spatially-Aware Rate-Guided Progressive Fine-tuning (SARP-FT). This instance-level strategy fine-tunes the model for each test image by optimizing low-rank adapters on progressively larger, spatially-continuous regions selected via estimated information density. Experiments on diverse datasets (natural, satellite, medical) validate that our method achieves new state-of-the-art compression. Notably, our approach sets a new benchmark in learned lossless compression, showing a carefully designed AR framework can offer significant gains over existing methods with a small parameter count and competitive coding speeds.

Problem

• Overcoming computational impracticality of autoregressive models for lossless image compression
• Enhancing compression efficiency through hierarchical parallelism and progressive adaptation
• Achieving state-of-the-art performance across diverse image datasets with lightweight design

Innovation

• Hierarchical parallelism in autoregressive models
• Progressive adaptation with low-rank fine-tuning
• Cache-based inference and adaptive focus coding