Determining Roe and Metz model parameters for simulating multireader multicase confidence-of-disease rating data based on real-data or conjectured Obuchowski–Rockette parameter estimates

Stephen L. Hillis; Brian J. Smith; Weijie Chen

doi:10.1117/1.JMI.9.4.045501

Back

Determining Roe and Metz model parameters for simulating multireader multicase confidence-of-disease rating data based on real-data or conjectured Obuchowski–Rockette parameter estimates

Journal article

Open access

Peer reviewed

Determining Roe and Metz model parameters for simulating multireader multicase confidence-of-disease rating data based on real-data or conjectured Obuchowski–Rockette parameter estimates

Stephen L. Hillis, Brian J. Smith and Weijie Chen

Journal of medical imaging (Bellingham, Wash.), Vol.9(4), pp.045501-045501

07/01/2022

DOI: 10.1117/1.JMI.9.4.045501

PMCID: PMC9268356

PMID: 35818569

Files and links (1)

url

https://doi.org/10.1117/1.JMI.9.4.045501View

Published (Version of record) Open Access

Abstract

Purpose: The most frequently used model for simulating multireader multicase (MRMC) data that emulates confidence-of-disease ratings from diagnostic imaging studies has been the Roe and Metz (RM) model, proposed by Roe and Metz in 1997 and later generalized by Hillis (2012), Abbey et al. (2013), and Gallas and Hillis (2014). A problem with these models is that it has been difficult to set model parameters such that the simulated data are similar to MRMC data encountered in practice. To remedy this situation, Hillis (2018) mapped parameters from the RM model to Obuchowski–Rockette (OR) model parameters that describe the distribution of the empirical AUC outcomes computed from the RM model simulated data. We continue that work by providing the reverse mapping, i.e., by deriving an algorithm that expresses RM parameters as functions of the OR empirical AUC distribution parameters. Approach: We solve for the corresponding RM parameters in terms of the OR parameters using numerical methods. Results: An algorithm is developed that results in, at most, one solution of RM parameter values that correspond to inputted OR parameter values. The algorithm can be implemented using an R software function. Examples are provided that illustrate the use of the algorithm. A simulation study validates the algorithm. Conclusions: The resulting algorithm makes it possible to easily determine RM model parameter values such that simulated data emulate a specific real-data study. Thus, MRMC analysis methods can be empirically tested using simulated data similar to that encountered in practice.

Obuchowski and Rockette

ROC curve

diagnostic radiology

Roe and Metz

simulated data

Details

Title: Subtitle: Determining Roe and Metz model parameters for simulating multireader multicase confidence-of-disease rating data based on real-data or conjectured Obuchowski–Rockette parameter estimates
Creators: Stephen L. Hillis - University of Iowa
Brian J. Smith - University of Iowa
Weijie Chen - Office of Science
Resource Type: Journal article
Publication Details: Journal of medical imaging (Bellingham, Wash.), Vol.9(4), pp.045501-045501
DOI: 10.1117/1.JMI.9.4.045501
PMID: 35818569
PMCID: PMC9268356
NLM abbreviation: J Med Imaging (Bellingham)
ISSN: 2329-4302
eISSN: 2329-4310
Publisher: Society of Photo-Optical Instrumentation Engineers
Grant note: R01EB025174 / National Institute of Biomedical Imaging and Bioengineering of the National Institutes of Health under
Language: English
Date published: 07/01/2022
Academic Unit: Radiology; Biostatistics; Holden Comprehensive Cancer Center
Record Identifier: 9984293560002771

Metrics

25 Record Views

2 Times Cited - Web of Science