Improving Accuracy and Efficiency of Graph Embedding Training with Fine-Grained Parameter Management

Lihan Hu; Peng Jiang

doi:10.1109/IPDPS64566.2025.00071

Back

Conference proceeding

Improving Accuracy and Efficiency of Graph Embedding Training with Fine-Grained Parameter Management

Lihan Hu and Peng Jiang

Proceedings - IEEE International Parallel and Distributed Processing Symposium, pp.737-748

International Parallel and Distributed Processing Symposium IPDPS

06/03/2025

DOI: 10.1109/IPDPS64566.2025.00071

View Online

Abstract

Efficient and accurate graph embedding learning is crucial for various real-world applications. However, the large-scale nature of graph embeddings poses significant challenges, particularly in managing the massive amount of embedding parameters across CPU and GPU memories. This paper presents a fine-grained parameter management technique that significantly improves the accuracy and efficiency of graph embedding learning. Our approach leverages parameter duplication and a novel precaching strategy, which minimizes the overhead of data movement between CPU and GPU while preventing stale data usage during training. We introduce an analytical model to estimate the access frequency of embeddings, allowing for the optimal placement of embedding data between CPU and GPU. Using a zero-copy data access mechanism, our system effectively reduces training time while maintaining high accuracy. Experimental results on multiple large-scale knowledge graphs demonstrate that our approach achieves substantial performance gains compared to existing methods with improved Mean Reciprocal Rank (MRR).

Accuracy

Analytical models

data movement

Frequency estimation

Graphics processing units

knowledge graph embedding

Knowledge graphs

Memory management

Performance gain

Radio spectrum management

Resource management

Training

Details

Title: Subtitle: Improving Accuracy and Efficiency of Graph Embedding Training with Fine-Grained Parameter Management
Creators: Lihan Hu - University of Iowa
Peng Jiang - University of Iowa
Resource Type: Conference proceeding
Publication Details: Proceedings - IEEE International Parallel and Distributed Processing Symposium, pp.737-748
Series: International Parallel and Distributed Processing Symposium IPDPS
DOI: 10.1109/IPDPS64566.2025.00071
ISSN: 1530-2075
eISSN: 1530-2075
Publisher: IEEE
Grant note: CNS-2310423 / NSF (10.13039/100000001)
Language: English
Date published: 06/03/2025
Academic Unit: Computer Science
Record Identifier: 9984927209102771

Metrics

5 Record Views