Communication efficient distributed learning with feature partitioned data

Bingwen Zhang; Jun Geng; Weiyu Xu; Lifeng Lai

doi:10.1109/CISS.2018.8362294

Back

Conference proceeding

Communication efficient distributed learning with feature partitioned data

Bingwen Zhang, Jun Geng, Weiyu Xu and Lifeng Lai

2018 52nd Annual Conference on Information Sciences and Systems (CISS), pp.1-6

03/2018

DOI: 10.1109/CISS.2018.8362294

View Online

Abstract

One major bottleneck in the design of large scale distributed machine learning algorithms is the communication cost. In this paper, we propose and analyze a distributed learning scheme for reducing the amount of communication in distributed learning problems under the feature partition scenario. The motivating observation of our scheme is that, in the existing schemes for the feature partition scenario, large amount of data exchange is needed for calculating gradients. In our proposed scheme, instead of calculating the exact gradient at each iteration, we only calculate the exact gradient sporadically. We provide precise conditions to determine when to perform the exact update, and characterize the convergence rate and bounds for total iterations and communication iterations. We further test our algorithm on real data sets and show that the proposed scheme can substantially reduce the amount of data transferred between distributed nodes.

Approximation algorithms

Communication efficiency

Convergence

Distributed databases

Distributed learning

Feature partitioned data

Inexact update

Loss measurement

Machine learning algorithms

Optimization

Partitioning algorithms

Details

Title: Subtitle: Communication efficient distributed learning with feature partitioned data
Creators: Bingwen Zhang - Dept. of Elec. and Comp. Engr., Worcester Poly. Inst., Worcester, MA
Jun Geng - Harbin Institute of Technology
Weiyu Xu - University of Iowa
Lifeng Lai - Dept. of Elec. and Comp. Engr., U. of California, Davis, CA
Resource Type: Conference proceeding
Publication Details: 2018 52nd Annual Conference on Information Sciences and Systems (CISS), pp.1-6
DOI: 10.1109/CISS.2018.8362294
Publisher: IEEE
Language: English
Date published: 03/2018
Academic Unit: Electrical and Computer Engineering
Record Identifier: 9984197293602771

Metrics

5 Record Views