Journal article
Multi-reader ROC studies with split-plot designs: a comparison of statistical methods
Academic radiology, Vol.19(12), pp.1508-1517
12/2012
DOI: 10.1016/j.acra.2012.09.012
PMCID: PMC3522484
PMID: 23122570
Abstract
Multireader imaging trials often use a factorial design, in which study patients undergo testing with all imaging modalities and readers interpret the results of all tests for all patients. A drawback of this design is the large number of interpretations required of each reader. Split-plot designs have been proposed as an alternative, in which one or a subset of readers interprets all images of a sample of patients, while other readers interpret the images of other samples of patients. In this paper, the authors compare three methods of analysis for the split-plot design.
Three statistical methods are presented: the Obuchowski-Rockette method modified for the split-plot design, a newly proposed marginal-mean analysis-of-variance approach, and an extension of the three-sample U-statistic method. A simulation study using the Roe-Metz model was performed to compare the type I error rate, power, and confidence interval coverage of the three test statistics.
The type I error rates for all three methods are close to the nominal level but tend to be slightly conservative. The statistical power is nearly identical for the three methods. The coverage of 95% confidence intervals falls close to the nominal coverage for small and large sample sizes.
The split-plot multireader, multicase study design can be statistically efficient compared to the factorial design, reducing the number of interpretations required per reader. Three methods of analysis, shown to have nominal type I error rates, similar power, and nominal confidence interval coverage, are available for this study design.
Details
- Title: Subtitle
- Multi-reader ROC studies with split-plot designs: a comparison of statistical methods
- Creators
- Nancy A Obuchowski - Department of Quantitative Health Sciences, JJN-3, Cleveland Clinic Foundation, 9500 Euclid Avenue, Cleveland, OH 44195, USA. obuchon@ccf.orgBrandon D GallasStephen L Hillis
- Resource Type
- Journal article
- Publication Details
- Academic radiology, Vol.19(12), pp.1508-1517
- DOI
- 10.1016/j.acra.2012.09.012
- PMID
- 23122570
- PMCID
- PMC3522484
- NLM abbreviation
- Acad Radiol
- ISSN
- 1076-6332
- eISSN
- 1878-4046
- Publisher
- United States
- Grant note
- R01EB000863 / NIBIB NIH HHS R01 EB000863 / NIBIB NIH HHS R01EB013667 / NIBIB NIH HHS R01 EB013667 / NIBIB NIH HHS
- Language
- English
- Date published
- 12/2012
- Academic Unit
- Radiology; Biostatistics
- Record Identifier
- 9984051702402771
Metrics
24 Record Views