Conference proceeding
ECCH: Erasure Coded Consistent Hashing for Distributed Storage Systems
The Institute of Electrical and Electronics Engineers, Inc. (IEEE) Conference Proceedings
01/01/2021
DOI: 10.1109/ISPA-BDCloud-SocialCom-SustainCom52081.2021.00036
Abstract
Conference Title: 2021 IEEE Intl Conf on Parallel & Distributed Processing with Applications, Big Data & Cloud Computing, Sustainable Computing & Communications, Social Computing & Networking (ISPA/BDCloud/SocialCom/SustainCom) Conference Start Date: 2021, Sept. 30 Conference End Date: 2021, Oct. 3 Conference Location: New York City, NY, USAIn this paper, we propose ECCH, an Erasure Coded Consistent Hashing scheme to make better data placement in distributed storage systems. It combines the inherent advantages of consistent hashing together with the storage-efficiency of erasure coding technology. Specifically, ECCH divides data block stream of files into groups according to block IDs. In each group, it encoded data blocks with additional parity blocks by erasure coding. All encoded blocks in the same group are stored on different nodes with consistent hashing distribution. For node failure or data loss, ECCH locates required data through the ID of missing block in a same group for fast recovery. To deal with node changes, ECCH introduces a design of multi-version hash rings to manage data layout. It can prevent the impact of data migration on erasure coding, while achieving data balance with little data movement. We have implemented ECCH on the Sheepdog, a distributed object-based storage system. Evaluation results show that ECCH can greatly improve the space utilization of hashing-based storage systems, while achieving efficient fault tolerance.
Details
- Title: Subtitle
- ECCH: Erasure Coded Consistent Hashing for Distributed Storage Systems
- Creators
- Jiang ZhouLin SuWeiping WangYong Chen
- Resource Type
- Conference proceeding
- Publication Details
- The Institute of Electrical and Electronics Engineers, Inc. (IEEE) Conference Proceedings
- DOI
- 10.1109/ISPA-BDCloud-SocialCom-SustainCom52081.2021.00036
- Publisher
- The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
- Grant note
- DOI: 10.13039/501100009592, name: Beijing Municipal Science and Technology Commission; DOI: 10.13039/501100002367, name: Chinese Academy of Sciences; DOI: 10.13039/100000001, name: National Science Foundation
- Language
- English
- Date published
- 01/01/2021
- Academic Unit
- Industrial and Systems Engineering
- Record Identifier
- 9984203259602771
Metrics
76 Record Views