Dissertation
Data volume reduction and efficient data placement for accelerator-based HPC systems
University of Iowa
Doctor of Philosophy (PhD), University of Iowa
Spring 2026
DOI: 10.25820/etd.008441
Abstract
As modern high-performance computing (HPC) and machine learning (ML) applications continue to grow in scale, data-centric bottlenecks increasingly dominate system performance. These bottlenecks include costly data movement as well as excessive memory and storage demands. This dissertation addresses these challenges by developing architecture-conscious techniques for efficient data placement and data volume reduction, with the goal of enabling more scalable execution of big-data workloads.
The first part of this dissertation studies data movement in large-scale graph neural network (GNN) training on multiple GPUs. Existing graph partitioning methods are not well aligned with sampled, data-parallel GNN training, where feature loading from CPU memory to GPUs becomes a major bottleneck. To address this issue, this dissertation develops a performance model for data movement between CPU and GPUs, proposes an efficient data placement strategy that minimizes data loading time, and introduces a locality-aware neighbor sampling technique to further reduce movement overhead while preserving training accuracy.
The second part of this dissertation focuses on scientific data compression for the Cerebras Wafer-Scale Engine (WSE), an emerging accelerator with a distributed dataflow architecture. As scientific applications continue to generate massive volumes of data, efficient compression becomes increasingly important for reducing storage and data transfer costs. However, prior error-bounded lossy compressors were primarily designed for conventional CPU and GPU platforms, while compression support for the Cerebras system remained largely unexplored. To address this gap, this dissertation presents CereSZ, the first end-to-end error-bounded lossy compression framework for the Cerebras CS-2 system. CereSZ establishes a compression pipeline tailored to the WSE architecture through block-wise design, stage-wise pipelining, and coordinated execution across processing elements, enabling both compression and decompression on the platform.
Building on this foundation, the dissertation then presents CereSZ-II, a memory-efficient and computation-balanced lossy compressor that addresses key limitations of the original design. In particular, CereSZ-II introduces a computation-balanced fixed-size Huffman encoding scheme to accommodate the limited local memory of processing elements, along with a spatial-aware offset computation mechanism for efficient compressed-data concatenation without shared memory. These techniques improve the practicality and scalability of lossy compression on wafer-scale systems and further advance the use of Cerebras for data-intensive scientific workloads.
Manually implementing optimized compression algorithms for platforms such as the Cerebras WSE is labor-intensive and requires substantial low-level hardware expertise. To address this programmability challenge, the last part of this dissertation presents P3Z, a domain-specific compiler for lossy compression for improving the portability and productivity of high-performance compressor design on modern accelerators. P3Z allows users to express compressors in a compact Python-based domain-specific language, lowers the program into an abstract syntax graph and intermediate representation (IR), applies IR-level and hardware-aware optimizations, and automatically generates optimized code for both CPU and Cerebras WSE backends. P3Z also supports semantics-aware automatic decompression, deriving decompression kernels directly from the compression pipeline.
Together, these contributions establish a unified framework for addressing data bottlenecks in modern accelerators. By co-designing algorithms, systems, and compiler techniques, this dissertation demonstrates that scalable data placement, efficient scientific data compression, and compiler-driven code generation can collectively enable more efficient execution of next-generation scientific and AI workloads.
Details
- Title: Subtitle
- Data volume reduction and efficient data placement for accelerator-based HPC systems
- Creators
- Shihui Song
- Contributors
- Peng Jiang (Advisor)Steve Goddard (Committee Member)Mehrdad Moharrami (Committee Member)Tianyu Zhang (Committee Member)Robert Underwood (Committee Member)
- Resource Type
- Dissertation
- Degree Awarded
- Doctor of Philosophy (PhD), University of Iowa
- Degree in
- Computer Science
- Date degree season
- Spring 2026
- DOI
- 10.25820/etd.008441
- Publisher
- University of Iowa
- Number of pages
- xv, 145 pages
- Copyright
- Copyright 2026 Shihui Song
- Language
- English
- Date submitted
- 04/20/2026
- Description illustrations
- Illustrations, graphs, charts, tables
- Description bibliographic
- Includes bibliographical references (pages 130-145).
- Public Abstract (ETD)
As modern high-performance computing (HPC) and machine learning (ML) applications continue to grow in scale, data-centric bottlenecks increasingly dominate system performance. These bottlenecks include costly data movement as well as excessive memory and storage demands. This thesis addresses these challenges by developing architecture-conscious techniques for efficient data placement and data volume reduction, with the goal of enabling more scalable execution of big-data workloads. First, it studies data movement in large-scale graph neural network training and proposes an efficient data placement strategy that minimizes data loading time when moving graph features between CPU memory and multiple GPUs. It then presents CereSZ, the first end-to-end error-bounded lossy compression framework for the Cerebras CS-2 system, to help reduce the storage and communication costs of data-intensive applications. CereSZ establishes a compression pipeline with block-wise design, stage-wise pipelining, and coordinated execution across processing elements. Building on this effort, it introduces CereSZ-II, which further improves the computational efficiency and scalability of lossy compression on Cerebras through a more efficient fixed-size Huffman encoding method. Finally, it presents P3Z, a domain-specific compiler that automates the generation of high-performance lossy compression code for both CPU and Cerebras platforms, improving programmability while preserving performance.
Together, these contributions show that improving how data is moved, reduced, and programmed is essential for making future scientific and AI applications more scalable, efficient, and practical on modern computing platforms.
- Academic Unit
- Computer Science
- Record Identifier
- 9985176871802771
Metrics
1 Record Views