Journal article
Analysis of model development strategies: predicting ventral hernia recurrence
The Journal of surgical research, Vol.206(1), pp.159-167
11/2016
DOI: 10.1016/j.jss.2016.07.042
PMID: 27916356
Abstract
There have been many attempts to identify variables associated with ventral hernia recurrence; however, it is unclear which statistical modeling approach results in models with greatest internal and external validity. We aim to assess the predictive accuracy of models developed using five common variable selection strategies to determine variables associated with hernia recurrence.
Two multicenter ventral hernia databases were used. Database 1 was randomly split into “development” and “internal validation” cohorts. Database 2 was designated “external validation”. The dependent variable for model development was hernia recurrence. Five variable selection strategies were used: (1) “clinical”—variables considered clinically relevant, (2) “selective stepwise”—all variables with a P value <0.20 were assessed in a step-backward model, (3) “liberal stepwise”—all variables were included and step-backward regression was performed, (4) “restrictive internal resampling,” and (5) “liberal internal resampling.” Variables were included with P < 0.05 for the Restrictive model and P < 0.10 for the Liberal model. A time-to-event analysis using Cox regression was performed using these strategies. The predictive accuracy of the developed models was tested on the internal and external validation cohorts using Harrell's C-statistic where C > 0.70 was considered “reasonable”.
The recurrence rate was 32.9% (n = 173/526; median/range follow-up, 20/1-58 mo) for the development cohort, 36.0% (n = 95/264, median/range follow-up 20/1-61 mo) for the internal validation cohort, and 12.7% (n = 155/1224, median/range follow-up 9/1-50 mo) for the external validation cohort. Internal validation demonstrated reasonable predictive accuracy (C-statistics = 0.772, 0.760, 0.767, 0.757, 0.763), while on external validation, predictive accuracy dipped precipitously (C-statistic = 0.561, 0.557, 0.562, 0.553, 0.560).
Predictive accuracy was equally adequate on internal validation among models; however, on external validation, all five models failed to demonstrate utility. Future studies should report multiple variable selection techniques and demonstrate predictive accuracy on external data sets for model validation.
Details
- Title: Subtitle
- Analysis of model development strategies: predicting ventral hernia recurrence
- Creators
- Julie L Holihan - Department of Surgery, University of Texas Health Science Center at Houston, Houston, TexasLinda T Li - Department of Surgery, Baylor College of Medicine, Houston, TexasErik P Askenasy - Department of Surgery, Baylor College of Medicine, Houston, TexasJacob A Greenberg - Department of Surgery, University of Wisconsin, Madison, WisconsinJerrod N Keith - Department of Surgery, University of Iowa, Iowa City, IowaRobert G Martindale - Department of Surgery, Oregon Health and Science University, Portland, OregonJ. Scott Roth - Department of Surgery, University of Kentucky, Lexington, KentuckyMike K Liang - Department of Surgery, University of Texas Health Science Center at Houston, Houston, Texas
- Resource Type
- Journal article
- Publication Details
- The Journal of surgical research, Vol.206(1), pp.159-167
- Publisher
- Elsevier Inc
- DOI
- 10.1016/j.jss.2016.07.042
- PMID
- 27916356
- ISSN
- 0022-4804
- eISSN
- 1095-8673
- Language
- English
- Date published
- 11/2016
- Academic Unit
- Surgery
- Record Identifier
- 9984051795602771
Metrics
20 Record Views