Journal article
The VAMPIRE challenge: A multi-institutional validation study of CT ventilation imaging
Medical physics (Lancaster), Vol.46(3), pp.1198-1217
03/2019
DOI: 10.1002/mp.13346
PMCID: PMC6605778
PMID: 30575051
Abstract
CT ventilation imaging (CTVI) is being used to achieve functional avoidance lung cancer radiation therapy in three clinical trials (NCT02528942, NCT02308709, NCT02843568). To address the need for common CTVI validation tools, we have built the Ventilation And Medical Pulmonary Image Registration Evaluation (VAMPIRE) Dataset, and present the results of the first VAMPIRE Challenge to compare relative ventilation distributions between different CTVI algorithms and other established ventilation imaging modalities.
The VAMPIRE Dataset includes 50 pairs of 4DCT scans and corresponding clinical or experimental ventilation scans, referred to as reference ventilation images (RefVIs). The dataset includes 25 humans imaged with Galligas 4DPET/CT, 21 humans imaged with DTPA-SPECT, and 4 sheep imaged with Xenon-CT. For the VAMPIRE Challenge, 16 subjects were allocated to a training group (with RefVI provided) and 34 subjects were allocated to a validation group (with RefVI blinded). Seven research groups downloaded the Challenge dataset and uploaded CTVIs based on deformable image registration (DIR) between the 4DCT inhale/exhale phases. Participants used DIR methods broadly classified into B-splines, Free-form, Diffeomorphisms, or Biomechanical modeling, with CT ventilation metrics based on the DIR evaluation of volume change, Hounsfield Unit change, or various hybrid approaches. All CTVIs were evaluated against the corresponding RefVI using the voxel-wise Spearman coefficient
, and Dice similarity coefficients evaluated for low function lung (
) and high function lung (
).
A total of 37 unique combinations of DIR method and CT ventilation metric were either submitted by participants directly or derived from participant-submitted DIR motion fields using the in-house software, VESPIR. The
and DSC results reveal a high degree of inter-algorithm and intersubject variability among the validation subjects, with algorithm rankings changing by up to ten positions depending on the choice of evaluation metric. The algorithm with the highest overall cross-modality correlations used a biomechanical model-based DIR with a hybrid ventilation metric, achieving a median (range) of 0.49 (0.27-0.73) for
, 0.52 (0.36-0.67) for
, and 0.45 (0.28-0.62) for
. All other algorithms exhibited at least one negative
value, and/or one DSC value less than 0.5.
The VAMPIRE Challenge results demonstrate that the cross-modality correlation between CTVIs and the RefVIs varies not only with the choice of CTVI algorithm but also with the choice of RefVI modality, imaging subject, and the evaluation metric used to compare relative ventilation distributions. This variability may arise from the fact that each of the different CTVI algorithms and RefVI modalities provides a distinct physiologic measurement. Ultimately this variability, coupled with the lack of a "gold standard," highlights the ongoing importance of further validation studies before CTVI can be widely translated from academic centers to the clinic. It is hoped that the information gleaned from the VAMPIRE Challenge can help inform future validation efforts.
Details
- Title: Subtitle
- The VAMPIRE challenge: A multi-institutional validation study of CT ventilation imaging
- Creators
- John Kipritidis - University of SydneyBilal A Tahir - University of SheffieldGuillaume Cazoulat - The University of Texas MD Anderson Cancer CenterMichael S Hofman - Peter MacCallum Cancer CentreShankar Siva - Peter MacCallum Cancer CentreJason Callahan - Peter MacCallum Cancer CentreNicholas Hardcastle - Peter MacCallum Cancer CentreTokihiro Yamamoto - University of California, DavisGary E Christensen - University of IowaJoseph M Reinhardt - University of IowaNoriyuki Kadoya - Tohoku UniversityTaylor J Patton - University of Wisconsin–MadisonSarah E Gerard - University of IowaIsabella Duarte - Duke UniversityBen Archibald-Heeren - University of WollongongMikel Byrne - Sydney Adventist HospitalRick Sims - Auckland Radiation Oncology, Auckland, New Zealand.Scott Ramsay - Auckland Radiation Oncology, Auckland, New Zealand.Jeremy T Booth - University of SydneyEnid Eslick - University of SydneyFiona Hegi-Johnson - Peter MacCallum Cancer CentreHenry C Woodruff - Maastricht UniversityRob H Ireland - University of SheffieldJim M Wild - University of SheffieldJing Cai - Hong Kong Polytechnic UniversityJohn E Bayouth - University of Wisconsin–MadisonKristy Brock - The University of Texas MD Anderson Cancer CenterPaul J Keall - University of Sydney
- Resource Type
- Journal article
- Publication Details
- Medical physics (Lancaster), Vol.46(3), pp.1198-1217
- DOI
- 10.1002/mp.13346
- PMID
- 30575051
- PMCID
- PMC6605778
- ISSN
- 0094-2405
- eISSN
- 2473-4209
- Grant note
- NIHR-RP-R3-12-027 / Department of Health Cancer Institute NSW Early Career Fellowship R01 HL079406 / NHLBI NIH HHS R01 HL112986 / NHLBI NIH HHS SP/14/6/31350 / British Heart Foundation P30 CA086862 / NCI NIH HHS R01HL079406 / National Institute of Health APP1060919 / Cancer Australia Priority-driven Collaborative Cancer Research Scheme R01 CA166703 / NCI NIH HHS P01 CA059827 / NCI NIH HHS R01CA166703 / National Institute of Health MR/M008894/1 / Medical Research Council P30 DK054759 / NIDDK NIH HHS
- Language
- English
- Date published
- 03/2019
- Academic Unit
- Roy J. Carver Department of Biomedical Engineering; Radiology; Electrical and Computer Engineering; Iowa Technology Institute; Radiation Oncology; Radiation Research Laboratory; The Iowa Institute for Biomedical Imaging; Advanced Pulmonary Physiomic Imaging Laboratory; Holden Comprehensive Cancer Center
- Record Identifier
- 9984197003102771
Metrics
21 Record Views