Journal article
A sparsity preserving stochastic gradient methods for sparse regression
Computational optimization and applications, Vol.58(2), pp.455-482
06/01/2014
DOI: 10.1007/s10589-013-9633-9
Abstract
We propose a new stochastic first-order algorithm for solving sparse regression problems. In each iteration, our algorithm utilizes a stochastic oracle of the subgradient of the objective function. Our algorithm is based on a stochastic version of the estimate sequence technique introduced by Nesterov (Introductory lectures on convex optimization: a basic course, Kluwer, Amsterdam, 2003). The convergence rate of our algorithm depends continuously on the noise level of the gradient. In particular, in the limiting case of noiseless gradient, the convergence rate of our algorithm is the same as that of optimal deterministic gradient algorithms. We also establish some large deviation properties of our algorithm. Unlike existing stochastic gradient methods with optimal convergence rates, our algorithm has the advantage of readily enforcing sparsity at all iterations, which is a critical property for applications of sparse regressions.
Details
- Title: Subtitle
- A sparsity preserving stochastic gradient methods for sparse regression
- Creators
- Qihang Lin - University of IowaXi Chen - University of California, BerkeleyJavier Pena - Carnegie Mellon University
- Resource Type
- Journal article
- Publication Details
- Computational optimization and applications, Vol.58(2), pp.455-482
- Publisher
- Springer Nature
- DOI
- 10.1007/s10589-013-9633-9
- ISSN
- 0926-6003
- eISSN
- 1573-2894
- Number of pages
- 28
- Language
- English
- Date published
- 06/01/2014
- Academic Unit
- Business Analytics
- Record Identifier
- 9984380391002771
Metrics
4 Record Views