Bayesian optimization of Random Forest hyperparameters for radiomics-based detection of clinically significant prostate cancer

Prostate cancer affects approximately one in eight men in the United States during their lifetime, making it one of the most common cancers in men. Early and accurate detection is crucial for successful treatment outcomes. Machine learning (ML) holds tremendous promise for improving detection rates and reducing diagnostic variability. However, implementing these technologies broadly in clinical settings has faced significant challenges.

This thesis presents a comprehensive case study of preparing a medical imaging ML system for prostate cancer detection from foundational assumptions. It addresses the fundamental issue that ML systems are only as effective as the data on which they are trained. Through systematic analysis of DICOM medical imaging data, ground truth standardization, and rigorous validation methodologies, this work demonstrates how untrustworthy data and undocumented assumptions can undermine ML performance in healthcare applications.

This research presents practical frameworks for ensuring data integrity, standardizing ground-truth annotations, and building maintainable machine learning training pipelines while addressing real-world clinical constraints. This work focuses on the process of improving an existing clinical data infrastructure for the purpose of ML refinements. This approach provides unique insights into challenges that are rarely addressed in academic literature but are commonly encountered in practice.

The findings underscore the crucial need to bridge the gap between theoretical advancements and clinical implementation. This work provides actionable methodologies for medical imaging professionals and ML developers to ensure that promising research innovations can effectively translate into technologies that benefit patients in clinical settings.