Conference proceeding
Communication efficient distributed learning with feature partitioned data
The Institute of Electrical and Electronics Engineers, Inc. (IEEE) Conference Proceedings, p.1
01/01/2018
Abstract
Conference Title: 2018 52nd Annual Conference on Information Sciences and Systems (CISS) Conference Start Date: 2018, March 21 Conference End Date: 2018, March 23 Conference Location: Princeton, NJ, USA One major bottleneck in the design of large scale distributed machine learning algorithms is the communication cost. In this paper, we propose and analyze a distributed learning scheme for reducing the amount of communication in distributed learning problems under the feature partition scenario. The motivating observation of our scheme is that, in the existing schemes for the feature partition scenario, large amount of data exchange is needed for calculating gradients. In our proposed scheme, instead of calculating the exact gradient at each iteration, we only calculate the exact gradient sporadically. We provide precise conditions to determine when to perform the exact update, and characterize the convergence rate and bounds for total iterations and communication iterations. We further test our algorithm on real data sets and show that the proposed scheme can substantially reduce the amount of data transferred between distributed nodes.
Details
- Title: Subtitle
- Communication efficient distributed learning with feature partitioned data
- Creators
- Bingwen ZhangJun GengWeiyu XuLifeng Lai
- Resource Type
- Conference proceeding
- Publication Details
- The Institute of Electrical and Electronics Engineers, Inc. (IEEE) Conference Proceedings, p.1
- Publisher
- The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
- Language
- English
- Date published
- 01/01/2018
- Academic Unit
- Electrical and Computer Engineering
- Record Identifier
- 9984198017702771
Metrics
5 Record Views