Journal article
Stochastic L-BFGS: Improved Convergence Rates and Practical Acceleration Strategies
IEEE transactions on signal processing, Vol.66(5), pp.1155-1169
03/01/2018
DOI: 10.1109/TSP.2017.2784360
Abstract
We revisit the stochastic limited-memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS) algorithm. By proposing a new coordinate transformation framework for the convergence analysis, we prove improved convergence rates and computational complexities of the stochastic L-BFGS algorithms compared to previous works. In addition, we propose several practical acceleration strategies to speed up the empirical performance of such algorithms. We also provide theoretical analyses for most of the strategies. Experiments on large-scale logistic and ridge regression problems demonstrate that our proposed strategies yield significant improvements vis-à-vis competing state-of-the-art algorithms.
Details
- Title: Subtitle
- Stochastic L-BFGS: Improved Convergence Rates and Practical Acceleration Strategies
- Creators
- Renbo Zhao - National University of SingaporeWilliam Benjamin Haskell - National University of SingaporeVincent Y. F. Tan - National University of Singapore
- Resource Type
- Journal article
- Publication Details
- IEEE transactions on signal processing, Vol.66(5), pp.1155-1169
- DOI
- 10.1109/TSP.2017.2784360
- ISSN
- 1053-587X
- eISSN
- 1941-0476
- Publisher
- IEEE
- Grant note
- R-263-000-C12-112 / MoE AcRF Tier 1 R-266-000-104-112 / MOE Tier I R-263-000-B37-133 / NUS Young Investigator
- Language
- English
- Date published
- 03/01/2018
- Academic Unit
- Business Analytics
- Record Identifier
- 9984446262002771
Metrics
12 Record Views