Journal article
Correlation of free-response and receiver-operating-characteristic area-under-the-curve estimates: Results from independently conducted FROC/ROC studies in mammography
Medical physics (Lancaster), Vol.39(10), pp.5917-5929
10/01/2012
DOI: 10.1118/1.4747262
PMCID: PMC3461051
PMID: 23039631
Abstract
Purpose: From independently conducted free-response receiver operating characteristic (FROC) and receiver operating characteristic (ROC) experiments, to study fixed-reader associations between three estimators: the area under the alternative FROC (AFROC) curve computed from FROC data, the area under the ROC curve computed from FROC highest rating data, and the area under the ROC curve computed from confidence-of-disease ratings.
Methods: Two hundred mammograms, 100 of which were abnormal, were processed by two image-processing algorithms and interpreted by four radiologists under the FROC paradigm. From the FROC data, inferred-ROC data were derived, using the highest rating assumption. Eighteen months afterwards, the images were interpreted by the same radiologists under the conventional ROC paradigm; conventional-ROC data (in contrast to inferred-ROC data) were obtained. FROC and ROC (inferred, conventional) data were analyzed using the nonparametric area-under-the-curve (AUC), (AFROC and ROC curve, respectively). Pearson correlation was used to quantify the degree of association between the modality-specific AUC indices and standard errors were computed using the bootstrap-after-bootstrap method. The magnitude of the correlations was assessed by comparison with computed Obuchowski-Rockette fixed reader correlations.
Results: Average Pearson correlations (with 95% confidence intervals in square brackets) were: Corr(FROC, inferred ROC) = 0.76[0.64, 0.84] > Corr(inferred ROC, conventional ROC) = 0.40[0.18, 0.58] > Corr (FROC, conventional ROC) = 0.32[0.16, 0.46].
Conclusions: Correlation between FROC and inferred-ROC data AUC estimates was high. Correlation between inferred- and conventional-ROC AUC was similar to the correlation between two modalities for a single reader using one estimation method, suggesting that the highest rating assumption might be questionable. (C) 2012 American Association of Physicists in Medicine. [http://dx.doi.org/10.1118/1.4747262]
Details
- Title: Subtitle
- Correlation of free-response and receiver-operating-characteristic area-under-the-curve estimates: Results from independently conducted FROC/ROC studies in mammography
- Creators
- Federica Zanca - KU LeuvenStephen L. Hillis - Iowa City VA Health Care SystemFilip Claus - KU LeuvenChantal Van Ongeval - KU LeuvenValerie Cells - Univ Hosp Leuven, Dept Radiol, B-3000 Louvain, BelgiumVeerle Provoost - KU LeuvenHong-Jun Yoon - University of PittsburghHilde Bosmans - KU Leuven
- Resource Type
- Journal article
- Publication Details
- Medical physics (Lancaster), Vol.39(10), pp.5917-5929
- DOI
- 10.1118/1.4747262
- PMID
- 23039631
- PMCID
- PMC3461051
- NLM abbreviation
- Med Phys
- ISSN
- 0094-2405
- eISSN
- 2473-4209
- Publisher
- Wiley
- Number of pages
- 13
- Grant note
- R01-EB005243; R01-EB008688 / Department of Health and Human Services, National Institutes of Health; United States Department of Health & Human Services; National Institutes of Health (NIH) - USA Mevic project R01EB013667 / NATIONAL INSTITUTE OF BIOMEDICAL IMAGING AND BIOENGINEERING; United States Department of Health & Human Services; National Institutes of Health (NIH) - USA; NIH National Institute of Biomedical Imaging & Bioengineering (NIBIB) R01-EB000863; R01-EB013667 / National Institutes of Health; United States Department of Health & Human Services; National Institutes of Health (NIH) - USA
- Language
- English
- Date published
- 10/01/2012
- Academic Unit
- Radiology; Biostatistics
- Record Identifier
- 9984318785502771
Metrics
11 Record Views