Search papers, labs, and topics across Lattice.
This paper addresses the problem of low data quality in pre-quantization based error-bounded lossy compressors, which are widely used for scientific data due to their high throughput. The authors characterize artifacts arising from pre-quantization by analyzing the correlation between quantization index and compression error. They then introduce a quantization-aware interpolation algorithm to improve decompressed data quality, demonstrating its effectiveness across five real-world datasets and two leading compressors while preserving high throughput via shared- and distributed-memory parallelization.
Pre-quantization compressors can get a serious data quality boost without sacrificing their speed, thanks to a new quantization-aware interpolation method.
Error-bounded lossy compression has been regarded as a promising way to address the ever-increasing amount of scientific data in today's high-performance computing systems. Pre-quantization, a critical technique to remove sequential dependency and enable high parallelism, is widely used to design and develop high-throughput error-controlled data compressors. Despite the extremely high throughput of pre-quantization based compressors, they generally suffer from low data quality with medium or large user-specified error bounds. In this paper, we investigate the artifacts generated by pre-quantization based compressors and propose a novel algorithm to mitigate them. Our contributions are fourfold: (1) We carefully characterize the artifacts in pre-quantization based compressors to understand the correlation between the quantization index and compression error; (2) We propose a novel quantization-aware interpolation algorithm to improve the decompressed data; (3) We parallelize our algorithm in both shared-memory and distributed-memory environments to obtain high performance; (4) We evaluate our algorithm and validate it with two leading pre-quantization based compressors using five real-world datasets. Experiments demonstrate that our artifact mitigation algorithm can effectively improve the quality of decompressed data produced by pre-quantization based compressors while maintaining their high compression throughput.
