Logo image
Back
Quantifying and leveraging predictive uncertainty for medical image assessment
Journal article   Peer reviewed

Quantifying and leveraging predictive uncertainty for medical image assessment

Florin C. Ghesu, Bogdan Georgescu, Awais Mansoor, Youngjin Yoo, Eli Gibson, R.S. Vishwanath, Abishek Balachandran, James M. Balter, Yue Cao, Ramandeep Singh, …
Medical image analysis, Vol.68, pp.101855-101855
02/2021
DOI: 10.1016/j.media.2020.101855
PMID: 33260116
url
https://arxiv.org/pdf/2007.04258View
Open Access

Abstract

•Quantification of predictive uncertainty using belief estimation and subjective logic.•Sample rejection based on predictive uncertainty leads to significant performance gain.•Predictive uncertainty correlates with multi-expert consensus decision.•Uncertainty-driven bootstrapping can improve system training and test performance. [Display omitted] The interpretation of medical images is a challenging task, often complicated by the presence of artifacts, occlusions, limited contrast and more. Most notable is the case of chest radiography, where there is a high inter-rater variability in the detection and classification of abnormalities. This is largely due to inconclusive evidence in the data or subjective definitions of disease appearance. An additional example is the classification of anatomical views based on 2D Ultrasound images. Often, the anatomical context captured in a frame is not sufficient to recognize the underlying anatomy. Current machine learning solutions for these problems are typically limited to providing probabilistic predictions, relying on the capacity of underlying models to adapt to limited information and the high degree of label noise. In practice, however, this leads to overconfident systems with poor generalization on unseen data. To account for this, we propose a system that learns not only the probabilistic estimate for classification, but also an explicit uncertainty measure which captures the confidence of the system in the predicted output. We argue that this approach is essential to account for the inherent ambiguity characteristic of medical images from different radiologic exams including computed radiography, ultrasonography and magnetic resonance imaging. In our experiments we demonstrate that sample rejection based on the predicted uncertainty can significantly improve the ROC-AUC for various tasks, e.g., by 8% to 0.91 with an expected rejection rate of under 25% for the classification of different abnormalities in chest radiographs. In addition, we show that using uncertainty-driven bootstrapping to filter the training data, one can achieve a significant increase in robustness and accuracy. Finally, we present a multi-reader study showing that the predictive uncertainty is indicative of reader errors.
 Belief estimation Building user trust Classification uncertainty Predictive uncertainty quantification Sample rejection Theory of evidence

Details

Metrics

7 Record Views
49 Times Cited - Web of Science
Logo image