Preprint
A Survey on Error-Bounded Lossy Compression for Scientific Datasets
arXiv.org
Cornell University
04/03/2024
DOI: 10.48550/arxiv.2404.02840
Abstract
Error-bounded lossy compression has been effective in significantly reducing
the data storage/transfer burden while preserving the reconstructed data
fidelity very well. Many error-bounded lossy compressors have been developed
for a wide range of parallel and distributed use cases for years. These lossy
compressors are designed with distinct compression models and design
principles, such that each of them features particular pros and cons. In this
paper we provide a comprehensive survey of emerging error-bounded lossy
compression techniques for different use cases each involving big data to
process. The key contribution is fourfold. (1) We summarize an insightful
taxonomy of lossy compression into 6 classic compression models. (2) We provide
a comprehensive survey of 10+ commonly used compression components/modules used
in error-bounded lossy compressors. (3) We provide a comprehensive survey of
10+ state-of-the-art error-bounded lossy compressors as well as how they
combine the various compression modules in their designs. (4) We provide a
comprehensive survey of the lossy compression for 10+ modern scientific
applications and use-cases. We believe this survey is useful to multiple
communities including scientific applications, high-performance computing,
lossy compression, and big data.
Details
- Title: Subtitle
- A Survey on Error-Bounded Lossy Compression for Scientific Datasets
- Creators
- Sheng DiJinyang LiuKai ZhaoXin LiangRobert UnderwoodZhaorui ZhangMilan ShahYafan HuangJiajun HuangXiaodong YuCongrong RenHanqi GuoGrant WilkinsDingwen TaoJiannan TianSian JinZizhe JianDaoce WangMD Hasanur RahmanBoyuan ZhangJon C CalhounGuanpeng LiKazutomo YoshiiKhalid Ayed AlharthiFranck Cappello
- Resource Type
- Preprint
- Publication Details
- arXiv.org
- DOI
- 10.48550/arxiv.2404.02840
- eISSN
- 2331-8422
- Publisher
- Cornell University; Ithaca, New York
- Language
- English
- Date posted
- 04/03/2024
- Academic Unit
- Computer Science
- Record Identifier
- 9984584907702771
Metrics
26 Record Views