Files and links (1)
Andres_Dajles_Thesis1.38 MB
Embargoed Access, Embargo ends: 06/26/2027
Dissertation
Quantifying model selection uncertainty: bootstrap-based measures for model comparisons and the evaluation of regression effects in multimodeling frameworks
University of Iowa
Doctor of Philosophy (PhD), University of Iowa
Spring 2025
DOI: 10.25820/etd.007880
Abstract
For statistical modeling applications, this dissertation develops and investigates inferential procedures that do not presuppose a model represents the truth, and that accommodate the uncertainty inherent in the model selection process.
Most practitioners of statistics heavily rely on p-values to decide between two competing models. However, traditional hypothesis testing requires that the null model is nested within a properly specified alternative, and the p-values are computed under the assumption that the null model represents the truth. Needless to say, such a framework is inherently limiting, and based on assumptions that are unrealistic in practice. In this work, we introduce measures that estimate the probability that a candidate null model is closer to the data-generating model than a competing alternative. The measures do not assume that either the null or the alternative model is true, and do not require the competing models to be nested.
In modeling applications involving multiple explanatory variables that can potentially characterize an outcome of interest, practitioners can select from a number of different candidate models based on various subsets of these explanatory factors. However, the most common practice is to choose one model from the candidate collection, and perform inference based on this final model, implicitly proceeding as though none of the other candidate models were ever considered. The consequences of ignoring model selection variability include potentially biased effect estimates, overly optimistic confidence intervals, and the failure to replicate results on different data sets representing the same phenomenon.
In this thesis, we propose a bootstrap-based multimodel approach for the analysis of effect estimates in the context of linear regression. We utilize the nonparametric bootstrap to conduct the model selection procedure multiple times. At each iteration, we consider the bootstrap effect estimate from the model being selected. The final, composite estimate is based on an average of all the estimates from these bootstrap selected models. Moreover, the bootstrap distribution of the replicated effect estimates allows us to compute the zero-value, which represents the probability that the estimated effect is equal to zero, and to build bootstrap percentile-based confidence intervals. The composite estimate, the zero-value and the bootstrap confidence intervals provide an inferential framework for linear regression that accounts for model selection variability.
Details
- Title: Subtitle
- Quantifying model selection uncertainty: bootstrap-based measures for model comparisons and the evaluation of regression effects in multimodeling frameworks
- Creators
- Andres David Dajles
- Contributors
- Joseph Cavanaugh (Advisor)Grant Brown (Committee Member)Gideon Zamba (Committee Member)Andrew Neath (Committee Member)
- Resource Type
- Dissertation
- Degree Awarded
- Doctor of Philosophy (PhD), University of Iowa
- Degree in
- Biostatistics
- Date degree season
- Spring 2025
- DOI
- 10.25820/etd.007880
- Publisher
- University of Iowa
- Number of pages
- xiii, 121 pages
- Copyright
- Copyright 2025 Andres David Dajles
- Language
- English
- Date submitted
- 10/31/2023
- Description illustrations
- illustrations, tables, graphs
- Description bibliographic
- Includes bibliographical references (pages 119-121).
- Public Abstract (ETD)
- When modeling data, practitioners often choose a model based on a combination of subject expertise and the execution of a model selection procedure. To assess the importance of the effects of interest, hypothesis testing and estimation are generally based on this favored model. Unfortunately, such inferences rely upon the assumption that the model under consideration subsumes the truth, which is rarely defensible in practical applications. Moreover, when multiple models are considered, the standard practice ignores the uncertainty inherent in selecting among the models from the candidate collection. In this work, for the comparison of two statistical models, we introduce a measure that approximates the probability that a candidate null model is closer to the truth than a competing alternative model. Unlike hypothesis testing, this measure does not assume that either model is an exact representation of reality, and does not require the competing models to be nested. Moreover, for applica- tions where multiple models are considered, we develop an inferential framework for performing inference on the effects of interest. This approach accounts for the variability inherent in selecting a model from a candidate collection.
- Academic Unit
- Biostatistics
- Record Identifier
- 9984830826402771
Metrics
2 Record Views