Mitigating Artifacts in Pre-quantization Based Scientific Data Compressors with Quantization-aware Interpolation

2026-02-23

AI Summary

This work addresses the significant artifacts introduced by pre-quantization compression under moderate to high error tolerances, which degrade the quality of decompressed data. For the first time, the study systematically characterizes the nature of pre-quantization artifacts and proposes a quantization-aware interpolation algorithm that substantially improves reconstruction accuracy within an error-bounded compression framework. The method tightly integrates quantization indices into the interpolation process and incorporates both shared- and distributed-memory parallel optimizations to maintain high throughput. Experimental evaluation on five real-world scientific datasets demonstrates that the proposed approach significantly enhances decompressed data fidelity while preserving the original compressor’s high throughput performance.

Technology Category

• Data Mining & Knowledge Management: Data Compression
• Machine Learning: Hardware-aware ML
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Querying, indexing, and retrieval in Web-related graphs
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Systems and Infrastructure for Web, Mobile and WoT: Data management and stream processing for Web, mobile and wireless applications

Abstract

Error-bounded lossy compression has been regarded as a promising way to address the ever-increasing amount of scientific data in today's high-performance computing systems. Pre-quantization, a critical technique to remove sequential dependency and enable high parallelism, is widely used to design and develop high-throughput error-controlled data compressors. Despite the extremely high throughput of pre-quantization based compressors, they generally suffer from low data quality with medium or large user-specified error bounds. In this paper, we investigate the artifacts generated by pre-quantization based compressors and propose a novel algorithm to mitigate them. Our contributions are fourfold: (1) We carefully characterize the artifacts in pre-quantization based compressors to understand the correlation between the quantization index and compression error; (2) We propose a novel quantization-aware interpolation algorithm to improve the decompressed data; (3) We parallelize our algorithm in both shared-memory and distributed-memory environments to obtain high performance; (4) We evaluate our algorithm and validate it with two leading pre-quantization based compressors using five real-world datasets. Experiments demonstrate that our artifact mitigation algorithm can effectively improve the quality of decompressed data produced by pre-quantization based compressors while maintaining their high compression throughput.

Problem

• pre-quantization
• scientific data compression
• artifacts
• error-bounded lossy compression
• data quality

Innovation

• pre-quantization
• quantization-aware interpolation
• error-bounded compression
• artifact mitigation
• parallel compression