Generalized Newton-Raphson algorithm for high dimensional LASSO regression

Yueyong Shi; Jian Huang; Yuling Jiao; Yicheng Kang; Hu Zhang

doi:10.4310/20-SII643

Back

Journal article

Generalized Newton-Raphson algorithm for high dimensional LASSO regression

Yueyong Shi, Jian Huang, Yuling Jiao, Yicheng Kang and Hu Zhang

Statistics and its interface, Vol.14(3), pp.339-350

01/01/2021

DOI: 10.4310/20-SII643

View Online

Abstract

The least absolute shrinkage and selection operator (LASSO) penalized regression is a state-of-the-art statistical method in high dimensional data analysis, when the number of predictors exceeds the number of observations. The commonly used Newton-Raphson algorithm is not very successful in solving the non-smooth optimization in LASSO. In this paper, we propose a fast generalized Newton-Raphson (GNR) algorithm for LASSO-type problems. The proposed algorithm, derived from a suitable Karush-Kuhn-Tucker (KKT) conditions based on generalized Newton derivatives, is a non-smooth Newton-type method. We first establish the local one-step convergence of GNR and then show that it is very efficient and accurate when coupled with a constinuation strategy. We also develop a novel parameter selection method. Numerical studies of simulated and real data analysis suggest that the GNR algorithm, with better (or comparable) accuracy, is faster than the algorithm implemented in the popular glmnet package.

Life Sciences & Biomedicine

Mathematical & Computational Biology

Mathematics

Mathematics, Interdisciplinary Applications

Physical Sciences

Science & Technology

Details

Title: Subtitle: Generalized Newton-Raphson algorithm for high dimensional LASSO regression
Creators: Yueyong Shi - China University of Geosciences
Jian Huang - University of Iowa
Yuling Jiao - Wuhan University
Yicheng Kang - Bentley University
Hu Zhang - Zhongnan University of Economics and Law
Resource Type: Journal article
Publication Details: Statistics and its interface, Vol.14(3), pp.339-350
Publisher: INT PRESS BOSTON, INC
DOI: 10.4310/20-SII643
ISSN: 1938-7989
eISSN: 1938-7997
Number of pages: 12
Grant note: 11871474 / National Natural Science Foundation of China; National Natural Science Foundation of China (NSFC) 17BTJ017 / National Social Science Fund of China
Language: English
Date published: 01/01/2021
Academic Unit: Statistics and Actuarial Science
Record Identifier: 9984257719802771

Metrics

41 Record Views