What to Support When You’re Compressing: The State of Practice Gaps and Opportunities for Scientific Data Compression

This research was supported by the Exascale Computing Project (ECP), Project Number: 17-SC-20-SC, a collaborative effort of two DOE organizations - the Office of Science and the National Nuclear Security Administration, responsible for the planning and preparation of a capable exascale ecosystem, including software, applications, hardware, advanced system engineering, and early testbed platforms, to support the nation's exascale computing imperative. The material was supported by the U.S. Department of Energy, Office of Science, Advanced Scientific Computing Research (ASCR), under contract DE-AC02-06CH11357, and supported by the National Science Foundation under Grant OAC-2003709/2303064, OAC-2104023/2247080, OAC-2311875/2311876/2311877, OAC-2312673, OAC-2034169, OAC-1751143, OAC-2330367, OAC-2313122, OAC-2311756, OIA-2327266 and OAC-2103621. We acknowledge the computing resources provided on Bebop (operated by the Laboratory Computing Resource Center at Argonne). Some of the experiments presented in this paper were carried out using the PlaFRIM experimental testbed, supported by Inria, CNRS (LABRI and IMB), Universite de Bordeaux, Bordeaux INP, and Conseil Regional d'Aquitaine (see https://www.plafrim.fr). TEZip's work has been supported by the COE research grant in computational science from Hyogo Prefecture and Kobe City through the Foundation for Computational Science. XIOS-SZ - Mario Acosta and Xavier Yepes-Arbos have received co-funding from the State Research Agency through OEMES (PID2020-116324RA-I00). We thank the Texas Advanced Computing Center (TACC) at the University of Texas at Austin for providing computational resources on the 'Frontera' system [55]. Use of the Linac Coherent Light Source (LCLS), SLAC National Accelerator Laboratory, is supported by the U.S. Department of Energy, Office of Science, Office of Basic Energy Sciences under Contract No. DEAC02-76SF00515. This work has been supported in part by the Department of Energy, Office of Science, under Award Number DE-SC0022223, as well as by equipment donations from NVIDIA Corporation. This work has been co-funded by the European Union through 'MDDB: Molecular Dynamics Data Bank. The European Repository for Biosimulation Data [101094651], and The Swedish e-Science Research Center.