Logo image
Back
Majorization Minimization by Coordinate Descent for Concave Penalized Generalized Linear Models
Journal article   Peer reviewed

Majorization Minimization by Coordinate Descent for Concave Penalized Generalized Linear Models

Dingfeng Jiang and Jian Huang
Statistics and computing, Vol.24(5), pp.871-883
09/2014
DOI: 10.1007/s11222-013-9407-3
PMCID: PMC4191872
PMID: 25309048
url
http://doi.org/10.1007/s11222-013-9407-3View
Open Access

Abstract

Recent studies have demonstrated theoretical attractiveness of a class of concave penalties in variable selection, including the smoothly clipped absolute deviation and minimax concave penalties. The computation of the concave penalized solutions in high-dimensional models, however, is a difficult task. We propose a majorization minimization by coordinate descent (MMCD) algorithm for computing the concave penalized solutions in generalized linear models. In contrast to the existing algorithms that use local quadratic or local linear approximation to the penalty function, the MMCD seeks to majorize the negative log-likelihood by a quadratic loss, but does not use any approximation to the penalty. This strategy makes it possible to avoid the computation of a scaling factor in each update of the solutions, which improves the efficiency of coordinate descent. Under certain regularity conditions, we establish theoretical convergence property of the MMCD. We implement this algorithm for a penalized logistic regression model using the SCAD and MCP penalties. Simulation studies and a data example demonstrate that the MMCD works sufficiently fast for the penalized logistic regression in high-dimensional settings where the number of covariates is much larger than the sample size.
 smoothly clipped absolute deviation penalty p ≫ n models variable selection logistic regression minimax concave penalty

Details

Metrics

29 Record Views
9 Times Cited - Web of Science
Logo image