JPEG Compliant Compression for Both Human and Machine, A Report

2025-03-13

AI Summary

JPEG compression exhibits a fundamental objective conflict between human visual perception and deep neural network (DNN) feature sensitivity. To address this, this paper proposes the first JPEG-standard-compliant joint image compression framework. We formulate compression as a multi-objective optimization problem, jointly minimizing perceptual distortion (for human observers) and task-specific accuracy degradation (for DNNs). We introduce the Human-Machine Optimal Error (HMOE) metric—a novel human-machine co-designed distortion measure—and develop the JPEG-compatible Human-Machine Optimal Soft Decision Quantization (HMOSDQ) scheme. HMOSDQ enables simultaneous optimization of rate–accuracy and rate–distortion trade-offs. Evaluated on an ImageNet subset, our method achieves a 0.81% absolute improvement in AlexNet top-1 validation accuracy at 0.61 bits per pixel (BPP) over standard JPEG; equivalently, it attains the same accuracy at a bitrate reduced by 9.6×. Moreover, it significantly enhances both subjective visual quality and model robustness.

Technology Category

• Computer Vision: Learning & Optimization for CV
• Machine Learning: Learning on the Edge & Model Compression
• Search and Optimization: Learning to Search

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Deep Neural Networks (DNNs) have become an integral part of our daily lives, especially in vision-related applications. However, the conventional lossy image compression algorithms are primarily designed for the Human Vision System (HVS), which can non-trivially compromise the DNNs' validation accuracy after compression, as noted in cite{liu2018deepn}. Thus developing an image compression algorithm for both human and machine (DNNs) is on the horizon. To address the challenge mentioned above, in this paper, we first formulate the image compression as a multi-objective optimization problem which take both human and machine prespectives into account, then we solve it by linear combination, and proposed a novel distortion measure for both human and machine, dubbed Human and Machine-Oriented Error (HMOE). After that, we develop Human And Machine Oriented Soft Decision Quantization (HMOSDQ) based on HMOE, a lossy image compression algorithm for both human and machine (DNNs), and fully complied with JPEG format. In order to evaluate the performance of HMOSDQ, finally we conduct the experiments for two pre-trained well-known DNN-based image classifiers named Alexnet cite{Alexnet} and VGG-16 cite{simonyan2014VGG} on two subsets of the ImageNet cite{deng2009imagenet} validation set: one subset included images with shorter side in the range of 496 to 512, while the other included images with shorter side in the range of 376 to 384. Our results demonstrate that HMOSDQ outperforms the default JPEG algorithm in terms of rate-accuracy and rate-distortion performance. For the Alexnet comparing with the default JPEG algorithm, HMOSDQ can improve the validation accuracy by more than $0.81%$ at $0.61$ BPP, or equivalently reduce the compression rate of default JPEG by $9.6 imes$ while maintaining the same validation accuracy.

Problem

• Develops JPEG-compliant image compression for both humans and DNNs.
• Proposes a novel distortion measure (HMOE) for human and machine perspectives.
• Outperforms default JPEG in rate-accuracy and rate-distortion performance.

Innovation

• Multi-objective optimization for human-machine image compression
• Human and Machine-Oriented Error (HMOE) distortion measure
• JPEG-compliant HMOSDQ algorithm for improved rate-accuracy performance