Journal article
Smoothing proximal gradient method for general structured sparse regression
The annals of applied statistics, Vol.6(2), pp.719-752
06/01/2012
DOI: 10.1214/11-AOAS514
Abstract
We study the problem of estimating high-dimensional regression models regularized by a structured sparsity-inducing penalty that encodes prior structural information on either the input or output variables. We consider two widely adopted types of penalties of this kind as motivating examples: (1) the general overlapping-group-lasso penalty, generalized from the group-lasso penalty; and (2) the graph-guided-fused-lasso penalty, generalized from the fused-lasso penalty. For both types of penalties, due to their nonseparability and nonsmoothness, developing an efficient optimization method remains a challenging problem. In this paper we propose a general optimization approach, the smoothing proximal gradient (SPG) method, which can solve structured sparse regression problems with any smooth convex loss under a wide spectrum of structured sparsity-inducing penalties. Our approach combines a smoothing technique with an effective proximal gradient method. It achieves a convergence rate significantly faster than the standard first-order methods, subgradient methods, and is much more scalable than the most widely used interior-point methods. The efficiency and scalability of our method are demonstrated on both simulation experiments and real genetic data sets.
Details
- Title: Subtitle
- Smoothing proximal gradient method for general structured sparse regression
- Creators
- Xi Chen - Carnegie Mellon UniversityQihang Lin - University of Iowa, Business AnalyticsSeyoung Kim - Carnegie Mellon Univ, Sch Comp Sci, Pittsburgh, PA 15213 USAJaime G. Carbonell - Carnegie Mellon Univ, Sch Comp Sci, Pittsburgh, PA 15213 USAEric P. Xing - Carnegie Mellon Univ, Sch Comp Sci, Pittsburgh, PA 15213 USA
- Resource Type
- Journal article
- Publication Details
- The annals of applied statistics, Vol.6(2), pp.719-752
- Publisher
- Inst Mathematical Statistics
- DOI
- 10.1214/11-AOAS514
- ISSN
- 1932-6157
- eISSN
- 1941-7330
- Number of pages
- 34
- Grant note
- Alfred P. Sloan Research Fellowship; Alfred P. Sloan Foundation
- Language
- English
- Date published
- 06/01/2012
- Academic Unit
- Business Analytics
- Record Identifier
- 9984380393702771
Metrics
17 Record Views