Conference proceeding
Hashing Based Data Distribution in Heterogeneous Storage
The Institute of Electrical and Electronics Engineers, Inc. (IEEE) Conference Proceedings
IEEE International Conference on Big Data and Cloud Computing (BdCloud), 19 (New York, New York, USA, 09/30/2021–10/03/2021)
01/01/2021
DOI: 10.1109/ISPA-BDCloud-SocialCom-SustainCom52081.2021.00095
Abstract
Conference Title: 2021 IEEE Intl Conf on Parallel & Distributed Processing with Applications, Big Data & Cloud Computing, Sustainable Computing & Communications, Social Computing & Networking (ISPA/BDCloud/SocialCom/SustainCom) Conference Start Date: 2021, Sept. 30 Conference End Date: 2021, Oct. 3 Conference Location: New York City, NY, USAStorage systems are important infrastructures of cloud computing in data centers. They demand the efficiency to distribute data and provide high I/O performance. The consistent hashing algorithm is widely used in modern storage systems due to its decentralized design, scalability, and adaptability to node changes. However, it lacks efficiency in a heterogeneous environment. In this study, we propose a Dynamically Attributed Consistent Hashing (DACH), to overcome this deficiency. DACH manages heterogeneous storage resources on consistent hashing ring and maintains multiple, dynamic attributes for nodes to characterize their distinct features. It places data on the ring and selects nodes with a balanced, weighted data distribution algorithm by taking full advantage of node attributes. By considering attribute variation, such as remaining capacity and workload changes, DACH can further optimize data layout. Extensive evaluation results show that, by well exploiting storage heterogeneity, DACH achieves adaptive and efficient data distribution for heterogeneous storage systems.
Details
- Title: Subtitle
- Hashing Based Data Distribution in Heterogeneous Storage
- Creators
- Lin SuWeiping WangYong Chen
- Resource Type
- Conference proceeding
- Publication Details
- The Institute of Electrical and Electronics Engineers, Inc. (IEEE) Conference Proceedings
- Conference
- IEEE International Conference on Big Data and Cloud Computing (BdCloud), 19 (New York, New York, USA, 09/30/2021–10/03/2021)
- DOI
- 10.1109/ISPA-BDCloud-SocialCom-SustainCom52081.2021.00095
- Publisher
- The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
- Grant note
- DOI: 10.13039/501100009592, name: Beijing Municipal Science and Technology Commission; DOI: 10.13039/501100002367, name: Chinese Academy of Sciences; DOI: 10.13039/100000001, name: National Science Foundation
- Language
- English
- Date published
- 01/01/2021
- Academic Unit
- Industrial and Systems Engineering
- Record Identifier
- 9984203259502771
Metrics
19 Record Views