On Degrees of Freedom of Projection Estimators With Applications to Multivariate Nonparametric Regression

Xi Chen; Qihang Lin; Bodhisattva Sen

doi:10.1080/01621459.2018.1537917

Back

On Degrees of Freedom of Projection Estimators With Applications to Multivariate Nonparametric Regression

Journal article

Peer reviewed

On Degrees of Freedom of Projection Estimators With Applications to Multivariate Nonparametric Regression

Xi Chen, Qihang Lin and Bodhisattva Sen

Journal of the American Statistical Association, Vol.115(529), pp.173-186

01/02/2020

DOI: 10.1080/01621459.2018.1537917

Files and links (1)

url

https://figshare.com/articles/journal_contribution/On_Degrees_of_Freedom_of_Projection_Estimators_With_Applications_to_Multivariate_Nonparametric_Regression/8026970View

Open Access

Abstract

Abstract-In this article, we consider the nonparametric regression problem with multivariate predictors. We provide a characterization of the degrees of freedom and divergence for estimators of the unknown regression function, which are obtained as outputs of linearly constrained quadratic optimization procedures; namely, minimizers of the least-squares criterion with linear constraints and/or quadratic penalties. As special cases of our results, we derive explicit expressions for the degrees of freedom in many nonparametric regression problems, for example, bounded isotonic regression, multivariate (penalized) convex regression, and additive total variation regularization. Our theory also yields, as special cases, known results on the degrees of freedom of many well-studied estimators in the statistics literature, such as ridge regression, Lasso and generalized Lasso. Our results can be readily used to choose the tuning parameter(s) involved in the estimation procedure by minimizing the Stein's unbiased risk estimate. As a by-product of our analysis we derive an interesting connection between bounded isotonic regression and isotonic regression on a general partially ordered set, which is of independent interest. Supplementary materials for this article are available online.

Additive model

Bounded isotonic regression

Divergence of an estimator

Generalized Lasso

Multivariate convex regression

Details

Title: Subtitle: On Degrees of Freedom of Projection Estimators With Applications to Multivariate Nonparametric Regression
Creators: Xi Chen - New York University
Qihang Lin - University of Iowa
Bodhisattva Sen - Columbia University
Resource Type: Journal article
Publication Details: Journal of the American Statistical Association, Vol.115(529), pp.173-186
DOI: 10.1080/01621459.2018.1537917
ISSN: 0162-1459
eISSN: 1537-274X
Publisher: Taylor & Francis
Grant note: DMS-1712822; AST-1614743 / NSF
Language: English
Date published: 01/02/2020
Academic Unit: Business Analytics
Record Identifier: 9984380429202771

Metrics

13 Record Views

16 Times Cited - Web of Science