Recent Developments in the Dorfman-Berbaum-Metz Procedure for Multireader ROC Study Analysis

Stephen L Hillis; Kevin S Berbaum; Charles E Metz

doi:10.1016/j.acra.2007.12.015

Back

Recent Developments in the Dorfman-Berbaum-Metz Procedure for Multireader ROC Study Analysis

Journal article

Peer reviewed

Recent Developments in the Dorfman-Berbaum-Metz Procedure for Multireader ROC Study Analysis

Stephen L Hillis, Kevin S Berbaum and Charles E Metz

Academic radiology, Vol.15(5), pp.647-661

2008

DOI: 10.1016/j.acra.2007.12.015

PMCID: PMC2737462

PMID: 18423323

View Online

Abstract

The Dorfman-Berbaum-Metz (DBM) method has been one of the most popular methods for analyzing multireader receiver-operating characteristic (ROC) studies since it was proposed in 1992. Despite its popularity, the original procedure has several drawbacks: it is limited to jackknife accuracy estimates, it is substantially conservative, and it is not based on a satisfactory conceptual or theoretical model. Recently, solutions to these problems have been presented in three papers. Our purpose is to summarize and provide an overview of these recent developments. We present and discuss the recently proposed solutions for the various drawbacks of the original DBM method. We compare the solutions in a simulation study and find that they result in improved performance for the DBM procedure. We also compare the solutions using two real data studies and find that the modified DBM procedure that incorporates these solutions yields more significant results and clearer interpretations of the variance component parameters than the original DBM procedure. We recommend using the modified DBM procedure that incorporates the recent developments.

jackknife

area under the curve (AUC)

Receiver-operating characteristic (ROC) curve

diagnostic radiology

DBM

Details

Title: Subtitle: Recent Developments in the Dorfman-Berbaum-Metz Procedure for Multireader ROC Study Analysis
Creators: Stephen L Hillis - Center for Research in the Implementation of Innovative Strategies in Practice (CRIISP), Iowa City VA Medical Center , 601 Highway 6 West, and Department of Biostatistics, University of Iowa, Iowa City, IA
Kevin S Berbaum - Department of Radiology, University of Iowa, Iowa City, IA
Charles E Metz - Department of Radiology, University of Chicago Medical Center, Chicago, IL
Resource Type: Journal article
Publication Details: Academic radiology, Vol.15(5), pp.647-661
DOI: 10.1016/j.acra.2007.12.015
PMID: 18423323
PMCID: PMC2737462
NLM abbreviation: Acad Radiol
ISSN: 1076-6332
eISSN: 1878-4046
Publisher: Elsevier BV
Language: English
Date published: 2008
Academic Unit: Radiology; Biostatistics
Record Identifier: 9984051544602771

Metrics

18 Record Views

183 Times Cited - Web of Science