Journal article
Multi-site quality and variability analysis of 3D FDG PET segmentations based on phantom and clinical image data
Medical physics (Lancaster), Vol.44(2), pp.479-496
02/2017
DOI: 10.1002/mp.12041
PMCID: PMC5834232
PMID: 28205306
Abstract
Radiomics utilizes a large number of image-derived features for quantifying tumor characteristics that can in turn be correlated with response and prognosis. Unfortunately, extraction and analysis of such image-based features is subject to measurement variability and bias. The challenge for radiomics is particularly acute in Positron Emission Tomography (PET) where limited resolution, a high noise component related to the limited stochastic nature of the raw data, and the wide variety of reconstruction options confound quantitative feature metrics. Extracted feature quality is also affected by tumor segmentation methods used to define regions over which to calculate features, making it challenging to produce consistent radiomics analysis results across multiple institutions that use different segmentation algorithms in their PET image analysis. Understanding each element contributing to these inconsistencies in quantitative image feature and metric generation is paramount for ultimate utilization of these methods in multi-institutional trials and clinical oncology decision making.
To assess segmentation quality and consistency at the multi-institutional level, we conducted a study of seven institutional members of the National Cancer Institute Quantitative Imaging Network. For the study, members were asked to segment a common set of phantom PET scans acquired over a range of imaging conditions as well as a second set of head and neck cancer (HNC) PET scans. Segmentations were generated at each institution using their preferred approach. In addition, participants were asked to repeat segmentations with a time interval between initial and repeat segmentation. This procedure resulted in overall 806 phantom insert and 641 lesion segmentations. Subsequently, the volume was computed from the segmentations and compared to the corresponding reference volume by means of statistical analysis.
On the two test sets (phantom and HNC PET scans), the performance of the seven segmentation approaches was as follows. On the phantom test set, the mean relative volume errors ranged from 29.9 to 87.8% of the ground truth reference volumes, and the repeat difference for each institution ranged between -36.4 to 39.9%. On the HNC test set, the mean relative volume error ranged between -50.5 to 701.5%, and the repeat difference for each institution ranged between -37.7 to 31.5%. In addition, performance measures per phantom insert/lesion size categories are given in the paper. On phantom data, regression analysis resulted in coefficient of variation (CV) components of 42.5% for scanners, 26.8% for institutional approaches, 21.1% for repeated segmentations, 14.3% for relative contrasts, 5.3% for count statistics (acquisition times), and 0.0% for repeated scans. Analysis showed that the CV components for approaches and repeated segmentations were significantly larger on the HNC test set with increases by 112.7% and 102.4%, respectively.
Analysis results underline the importance of PET scanner reconstruction harmonization and imaging protocol standardization for quantification of lesion volumes. In addition, to enable a distributed multi-site analysis of FDG PET images, harmonization of analysis approaches and operator training in combination with highly automated segmentation methods seems to be advisable. Future work will focus on quantifying the impact of segmentation variation on radiomics system performance.
Details
- Title: Subtitle
- Multi-site quality and variability analysis of 3D FDG PET segmentations based on phantom and clinical image data
- Creators
- Reinhard R Beichel - Department of Internal Medicine, The University of Iowa, Iowa City, IA, USABrian J Smith - Department of Biostatistics, The University of Iowa, Iowa City, IA, USAChristian Bauer - Department of Electrical and Computer Engineering, The University of Iowa, Iowa City, IA, USAEthan J Ulrich - Department of Biomedical Engineering, The University of Iowa, Iowa City, IA, USAPayam Ahmadvand - School of Computing Science, Simon Fraser University, Burnaby, CanadaMikalai M Budzevich - H Lee Moffitt Cancer Center, Tampa, FL, USARobert J Gillies - H Lee Moffitt Cancer Center, Tampa, FL, USADmitry Goldgof - Department of Computer Science and Engineering, University of South Florida, Tampa, FL, USAMilan Grkovski - Department of Medical Physics, Memorial Sloan Kettering Cancer Center, New York, NY, USAGhassan Hamarneh - School of Computing Science, Simon Fraser University, Burnaby, CanadaQiao Huang - Department of Radiology, Columbia University Medical Center, New York, NY, USAPaul E Kinahan - Department of Radiology, University of Washington Medical Center, Seattle, WA, USACharles M Laymon - Department of Radiology, University of Pittsburgh, Pittsburgh, PA, USAJames M Mountz - Department of Radiology, University of Pittsburgh, Pittsburgh, PA, USAJohn P Muzi - Department of Radiology, University of Washington Medical Center, Seattle, WA, USAMark Muzi - Department of Radiology, University of Washington Medical Center, Seattle, WA, USASadek Nehmeh - National Center for Cancer Care and Research, Doha, QatarMatthew J Oborski - Department of Bioengineering, University of Pittsburgh, Pittsburgh, PA, USAYongqiang Tan - Department of Radiology, Columbia University Medical Center, New York, NY, USABinsheng Zhao - Department of Radiology, Columbia University Medical Center, New York, NY, USAJohn J Sunderland - Department of Radiology, The University of Iowa, Iowa City, IA, USAJohn M Buatti - Department of Radiation Oncology, The University of Iowa, Iowa City, IA, USA
- Resource Type
- Journal article
- Publication Details
- Medical physics (Lancaster), Vol.44(2), pp.479-496
- Publisher
- United States
- DOI
- 10.1002/mp.12041
- PMID
- 28205306
- PMCID
- PMC5834232
- ISSN
- 0094-2405
- eISSN
- 2473-4209
- Grant note
- P30 ES005605 / NIEHS NIH HHS U01 CA200464 / NCI NIH HHS U01 CA140230 / NCI NIH HHS U01 CA143062 / NCI NIH HHS U01 CA157442 / NCI NIH HHS U01 CA140206 / NCI NIH HHS R01 CA169072 / NCI NIH HHS P30 CA086862 / NCI NIH HHS U01 CA148131 / NCI NIH HHS U24 CA180927 / NCI NIH HHS U24 CA180918 / NCI NIH HHS U01 CA140207 / NCI NIH HHS P30 CA047904 / NCI NIH HHS R50 CA211270 / NCI NIH HHS
- Language
- English
- Date published
- 02/2017
- Academic Unit
- Radiology; Electrical and Computer Engineering; Biostatistics; Physics and Astronomy; Radiation Oncology; Neurosurgery; Otolaryngology; Holden Comprehensive Cancer Center
- Record Identifier
- 9983997309502771
Metrics
18 Record Views