Model-based deep learning for medical image segmentation with global optimality

Image semantic segmentation, which partitions images into multiple segments on the pixel level, plays a fundamental role in computer vision applications. With superior data representation learning capacity, deep learning (DL) methods upraise as a dominant new generation of image segmentation alternatives with remarkable performance improvements over traditional algorithms. However, a generic deep learning network does not guarantee or enforce that its solution is globally optimal in feature space. This research aims to explore incorporating into deep learning the traditional continuous or discrete optimization models (interior-point method, differentiable dynamic programming, and min-cut) with domain knowledge constraints to achieve a global optimization solution in feature space for medical image surface and region semantic segmentation. These model-based deep learning networks can benefit from partial domain knowledge and learning from limited data. Experiments on public data sets of optical coherence tomography and diabetic foot ulcer images demonstrated that introducing these models further improves segmentation accuracy while guaranteeing global optimality. This study also presented a new data augmentation method in severe glaucoma disease application to enrich the variety of retinal layer patterns by 3D information reconstructing in optical coherence tomography images when the labeled severe disease data are rare. This study finally investigated an association between retinal layer thickness and arterial hypertension. It found that three retinal layer thicknesses were inversely correlated with higher blood pressure, and one retinal thickness was positively related to higher blood pressure. These association findings may be helpful for the detection of retinal diseases.