Journal article
Model selection criteria based on Kullback information measures for nonlinear regression
Journal of statistical planning and inference, Vol.134(2), pp.332-349
2005
DOI: 10.1016/j.jspi.2004.05.002
Abstract
In statistical modeling, selecting an optimal model from a class of candidates is a critical issue. During the past three decades, a number of model selection criteria have been proposed based on estimating Kullback's (Information Theory and Statistics, Dover, Mineola, NY, 1968, p. 5) directed divergence between the model generating the data and a fitted candidate model. The Akaike (Second International Symposium on Information Theory, Akadémia Kiadó, Budapest, Hungary, 1973, pp. 267–281; IEEE Trans. Automat. Control AC-19 (1974) 716) information criterion, AIC, was the first of these. AIC is justified in a very general framework, and as a result, offers a crude estimator of the directed divergence: one which exhibits a potentially high degree of negative bias in small-sample applications (Biometrika 76 (1989) 297). The “corrected” Akaike information criterion (Biometrika 76 (1989) 297), AICc, adjusts for this bias, and consequently often outperforms AIC as a selection criterion. However, AICc is less broadly applicable than AIC since its justification depends upon the structure of the candidate model.
AIC
I
(Biometrika 77 (1990) 709) is an “improved” version of AIC featuring a simulated bias correction.
Recently, model selection criteria have been proposed based on estimating Kullback's (Information Theory and Statistics, Dover, Mineola, NY, 1986, p. 6) symmetric divergence between the generating model and a fitted candidate model (Statist. Probab. Lett. 42 (1999) 333; Austral. New Zealand J. Statist. 46 (2004) 257). KIC, KICc, and
KIC
I
are criteria devised to target the symmetric divergence in the same manner that AIC, AICc, and
AIC
I
target the directed divergence.
AICc has been justified for the nonlinear regression framework by Hurvich and Tsai (Biometrika 76 (1989) 297). In this paper, we justify KICc for this framework, and propose versions of
AIC
I
and
KIC
I
suitable for nonlinear regression applications. We evaluate the selection performance of AIC, AICc,
AIC
I
, KIC, KICc, and
KIC
I
in a simulation study. Our results generally indicate that the “improved” criteria outperform the “corrected” criteria, which in turn outperform the non-adjusted criteria. Moreover, the KIC family performs favorably against the AIC family.
Details
- Title: Subtitle
- Model selection criteria based on Kullback information measures for nonlinear regression
- Creators
- Hyun-Joo Kim - Department of Mathematics and Computer Science, Truman State University, USAJoseph E Cavanaugh - Department of Biostatistics, C22-GH, College of Public Health, University of Iowa, 200 Hawkins Drive, Iowa City, IA 52242-1009, USA
- Resource Type
- Journal article
- Publication Details
- Journal of statistical planning and inference, Vol.134(2), pp.332-349
- DOI
- 10.1016/j.jspi.2004.05.002
- ISSN
- 0378-3758
- eISSN
- 1873-1171
- Publisher
- Elsevier B.V
- Language
- English
- Date published
- 2005
- Academic Unit
- Statistics and Actuarial Science; Biostatistics; Injury Prevention Research Center
- Record Identifier
- 9984214840702771
Metrics
14 Record Views