NIC-RobustBench: A Comprehensive Open-Source Toolkit for Neural Image Compression and Robustness Analysis

2025-06-23

AI Summary

Current neural image compression (NIC) methods suffer from limited adversarial robustness evaluation due to narrow codec coverage, insufficient attack diversity, and the absence of a unified benchmark. To address these issues, this paper introduces NIC-Robust—the first open-source robustness evaluation framework for NIC. It systematically integrates 12 state-of-the-art NIC models, supports eight representative adversarial attack types and five defense strategies, and enables joint assessment across rate-distortion-robustness trade-offs. NIC-Robust is the first to incorporate NIC robustness analysis into the JPEG AI standardization paradigm, providing automated testing pipelines, reproducible benchmark results, and modular extension interfaces. The framework is publicly released with open-source code, significantly advancing standardization and comparability in security-aware compression algorithm evaluation, and delivering empirical foundations for safety-critical next-generation image compression standards.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Adversarial Learning & Robustness
• Natural Language Processing: Safety and Robustness

Application Category

• Security and Privacy: Large-scale security measurements
• Responsible Web: Measurement, analysis, and circumvention of Web censorship
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Adversarial robustness of neural networks is an increasingly important area of research, combining studies on computer vision models, large language models (LLMs), and others. With the release of JPEG AI -- the first standard for end-to-end neural image compression (NIC) methods -- the question of evaluating NIC robustness has become critically significant. However, previous research has been limited to a narrow range of codecs and attacks. To address this, we present extbf{NIC-RobustBench}, the first open-source framework to evaluate NIC robustness and adversarial defenses' efficiency, in addition to comparing Rate-Distortion (RD) performance. The framework includes the largest number of codecs among all known NIC libraries and is easily scalable. The paper demonstrates a comprehensive overview of the NIC-RobustBench framework and employs it to analyze NIC robustness. Our code is available online at https://github.com/msu-video-group/NIC-RobustBench.

Problem

• Evaluating robustness of neural image compression methods
• Addressing limited codecs and attacks in NIC research
• Providing scalable framework for NIC robustness analysis

Innovation

• Open-source framework for NIC robustness evaluation
• Largest collection of codecs in NIC libraries
• Scalable design for comprehensive robustness analysis