Journal article
Distributed Dual Coordinate Ascent in General Tree Networks and Communication Network Effect on Synchronous Machine Learning
IEEE journal on selected areas in communications, Vol.39(7), pp.2105-2119
07/2021
DOI: 10.1109/JSAC.2021.3078495
Abstract
Due to the big size of data and limited data storage volume of a single computer or a single server, data are often stored in a distributed manner. Thus, performing large-scale machine learning operations with the distributed datasets through communication networks is often required. In this paper, we study the convergence rate of the distributed dual coordinate ascent for distributed machine learning problems in a general tree-structured network. Since a tree network model can be understood as the generalization of a star network, our algorithm can be thought of as the generalization of the distributed dual coordinate ascent in a star network. We provide the convergence rate of the distributed dual coordinate ascent over a general tree network in a recursive manner and analyze the network effect on the convergence rate. Secondly, by considering network communication delays, we optimize the distributed dual coordinate ascent algorithm to maximize its convergence speed. From our analytical result, we can choose the optimal number of local iterations depending on the communication delay severity to achieve the fastest convergence speed. In numerical experiments, we consider machine learning scenarios over communication networks, where local workers cannot directly reach to a central node due to constraints in communication, and demonstrate that the usability of our distributed dual coordinate ascent algorithm in tree networks.
Details
- Title: Subtitle
- Distributed Dual Coordinate Ascent in General Tree Networks and Communication Network Effect on Synchronous Machine Learning
- Creators
- Myung Cho - Penn State BehrendLifeng Lai - University of California, DavisWeiyu Xu - University of Iowa
- Resource Type
- Journal article
- Publication Details
- IEEE journal on selected areas in communications, Vol.39(7), pp.2105-2119
- Publisher
- IEEE
- DOI
- 10.1109/JSAC.2021.3078495
- ISSN
- 0733-8716
- eISSN
- 1558-0008
- Grant note
- ECCS-2000425 / NSF (10.13039/100000001) CCF-1717943; ECCS-2000415 / National Science Foundation (NSF) (10.13039/100000001)
- Language
- English
- Date published
- 07/2021
- Academic Unit
- Electrical and Computer Engineering
- Record Identifier
- 9984197199902771
Metrics
4 Record Views