Conference proceeding
MATLAB toolbox for ROC analysis of multi-reader multi-case diagnostic imaging studies
Medical Imaging 2022: Image Perception, Observer Performance, and Technology Assessment, Vol.12035, pp.120350G-120350G-13
04/04/2022
DOI: 10.1117/12.2610663
PMCID: PMC9504162
PMID: 36159880
Abstract
A common study design for comparing the performances of diagnostic imaging tests is to obtain ratings from multiple readers of multiple cases whose true statuses are known. Typically, there is overlap between the tests, readers, and/or cases for which special analytical methods are needed to perform statistical comparisons. We present our new MATLAB MRMCaov toolbox, which is designed for multi-reader multi-case comparisons of two or more diagnostic tests. The toolbox allows for statistical comparison of reader performance metrics, such as area under the receiver operating characteristic curve (ROC AUC), with analysis of variance methods originally proposed by Obuchowski and Rockette (1995) and later unified and improved by Hillis and colleagues (2005, 2007, 2008, 2018). MRMCaov is open-source software with an integrated command-line interface for performing multi-reader multi-case statistical analysis, plotting, and presenting results. Its features (1) ROC AUC, likelihood ratios of positive or negative ratings, sensitivity, specificity, and expected utility reader performance metrics; (2) reader-specific ROC curves; (3) user-definable performance metrics; (4) test-specific estimates of mean performance along with confidence intervals and p-values for statistical comparisons; (5) support for factorial, nested, or partially paired study designs; (6) inference for random or fixed readers and cases; (7) DeLong, jackknife, or unbiased covariance estimation; and (8) compatibility with Microsoft Windows, Mac OS, and Linux.
Details
- Title: Subtitle
- MATLAB toolbox for ROC analysis of multi-reader multi-case diagnostic imaging studies
- Creators
- Brian J. Smith - University of IowaStephen L. Hillis - University of Iowa
- Contributors
- Claudia R. Mello-Thoms (Editor) - Univ. Iowa Carver College of Medicine (United States)Sian Taylor-Phillips (Editor) - University of Warwick
- Resource Type
- Conference proceeding
- Publication Details
- Medical Imaging 2022: Image Perception, Observer Performance, and Technology Assessment, Vol.12035, pp.120350G-120350G-13
- DOI
- 10.1117/12.2610663
- PMID
- 36159880
- PMCID
- PMC9504162
- ISSN
- 1605-7422
- Publisher
- SPIE
- Language
- English
- Date published
- 04/04/2022
- Academic Unit
- Roy J. Carver Department of Biomedical Engineering; Radiology; Biostatistics; Holden Comprehensive Cancer Center
- Record Identifier
- 9984259955002771
Metrics
344 Record Views