Journal article
Honest leave‐one‐out cross‐validation for estimating post‐tuning generalization error
Stat (International Statistical Institute), Vol.10(1), e413
12/2021
DOI: 10.1002/sta4.413
Abstract
Many machine learning models have tuning parameters to be determined by the training data, and cross-validation (CV) is perhaps the most commonly used method for selecting tuning parameters. This work concerns the problem of estimating the generalization error of a CV-tuned predictive model. We propose to use an honest leave-one-out cross-validation framework to produce a nearly unbiased estimator of the post-tuning generalization error. By using the kernel support vector machine and the kernel logistic regression as examples, we demonstrate that the honest leave-one-out cross-validation has very competitive performance even when competing with the state-of-the-art .632+ estimator.
Details
- Title: Subtitle
- Honest leave‐one‐out cross‐validation for estimating post‐tuning generalization error
- Creators
- Boxiang Wang - University of IowaHui Zou - University of Minnesota
- Resource Type
- Journal article
- Publication Details
- Stat (International Statistical Institute), Vol.10(1), e413
- DOI
- 10.1002/sta4.413
- ISSN
- 2049-1573
- eISSN
- 2049-1573
- Grant note
- DOI: 10.13039/100000001, name: National Science Foundation, award: 1915‐842, 2015‐120
- Language
- English
- Date published
- 12/2021
- Academic Unit
- Statistics and Actuarial Science
- Record Identifier
- 9984257632702771
Metrics
7 Record Views