Dissertation
Efficient computation for Bayesian model averaging in linear regression models with heavy-tailed errors
University of Iowa
Doctor of Philosophy (PhD), University of Iowa
Spring 2026
DOI: 10.25820/etd.008450
Abstract
This thesis aims to develop Bayesian model averaging techniques in linear regression models to accommodate heavier-tailed error densities than the normal distribution, a scenario prevalent in economic and biomedical data. We primarily focus on the hyperbolic and Student-t distributions for modeling the errors.
Motivated by the use of the Huber loss function in the presence of outliers, the Bayesian Huberized lasso with hyperbolic errors has been proposed and recently implemented in the literature. Since the Huberized lasso cannot enforce regression coefficients to be exactly zero, we initially propose a fully Bayesian variable selection approach with spike-and-slab priors to address sparsity more effectively. Furthermore, the hyperbolic distribution has heavier tails than a normal distribution but thinner tails than a Cauchy distribution. Thus, we propose a novel regression model with an error distribution encompassing both hyperbolic and Student-t distributions. Our model aims to capture the benefit of using Huber loss, while adapting to heavier tails and unknown levels of sparsity, as entailed by the data. An efficient Gibbs sampler with Metropolis Hastings steps is developed for posterior computation.
For large model spaces, the potential entrapment of Markov chain Monte Carlo (MCMC) based methods with spike-and-slab priors poses significant challenges in posterior computation in regression models. On the other hand, maximum a posteriori (MAP) estimation, which is a more computationally viable alternative, fails to provide uncertainty quantification. To address these problems simultaneously and efficiently, we propose a hybrid method that blends MAP estimation with MCMC-based stochastic search algorithms within a heavy-tailed error framework. Under hyperbolic errors, we develop a two-step expectation conditional maximization (ECM) guided MCMC algorithm. In the first step, we conduct an ECM-based posterior maximization and perform variable selection, thereby identifying a reduced model space in a high posterior probability region. In the second step, we execute a Gibbs sampler on the reduced model space for posterior computation. Such a method is expected to improve the efficiency of posterior computation and enhance its inferential richness.
Through comparisons with various state-of-the-art methods in simulation studies and benchmark real life examples, both our proposed methods are shown to exhibit competitive performance and several advantages in parameter estimation, variable selection, point prediction and uncertainty quantification.
Details
- Title: Subtitle
- Efficient computation for Bayesian model averaging in linear regression models with heavy-tailed errors
- Creators
- Shamriddha De
- Contributors
- Joyee Ghosh (Advisor)Joseph Cavanaugh (Committee Member)Sanvesh Srivastava (Committee Member)Aixin Tan (Committee Member)
- Resource Type
- Dissertation
- Degree Awarded
- Doctor of Philosophy (PhD), University of Iowa
- Degree in
- Statistics
- Date degree season
- Spring 2026
- DOI
- 10.25820/etd.008450
- Publisher
- University of Iowa
- Number of pages
- xiii, 85 pages
- Copyright
- Copyright 2026 Shamriddha De
- Language
- English
- Date submitted
- 04/18/2026
- Description illustrations
- Illustrations, graphs, charts, tables
- Description bibliographic
- Includes bibliographical references (pages 81-85).
- Public Abstract (ETD)
Linear regression is a fundamental statistical tool for predicting an outcome variable from a set of explanatory variables, or covariates. For a large number of covariates, most of which might be redundant, it is necessary to identify the relevant ones, a task commonly referred to as variable selection. Such a problem of variable selection in linear regression has served as a classical inquiry in both the well-known statistical paradigms, namely, frequentist and Bayesian. However, traditional predictive models rely on normal error distributions to capture variability in data, an assumption which is sensitive to extreme observations/tail heaviness and often compromises with the modeling performance. Such datasets are common, especially in financial sectors pertaining to housing prices and salaries, as well as in biomedical sectors including gene-expression data. Consequently, there is a need to explore more robust techniques to handle the critical issue of tail heaviness. The goal of this thesis is to develop computationally efficient linear regression modeling techniques, which simultaneously address variable selection and modeling flexibility/robustness in a Bayesian framework.
- Academic Unit
- Statistics and Actuarial Science
- Record Identifier
- 9985177376302771
Metrics
1 Record Views