Logo image
Back
Invasive or More Direct Measurements Can Provide an Objective Early-Stopping Ceiling for Training Deep Neural Networks on Non-invasive or Less-Direct Biomedical Data
Journal article   Open access   Peer reviewed

Invasive or More Direct Measurements Can Provide an Objective Early-Stopping Ceiling for Training Deep Neural Networks on Non-invasive or Less-Direct Biomedical Data

Christopher W. Bartlett, Jamie Bossenbroek, Yukie Ueyama, Patricia McCallinhart, Olivia A. Peters, Donna A. Santillan, Mark K. Santillan, Aaron J. Trask and William C. Ray
SN computer science, Vol.4(2), 161
01/12/2023
DOI: 10.1007/s42979-022-01553-8
PMCID: PMC9836982
PMID: 36647373
url
https://doi.org/10.1007/s42979-022-01553-8View
Published (Version of record) Open Access

Abstract

Early stopping is an extremely common tool to minimize overfitting, which would otherwise be a cause of poor generalization of the model to novel data. However, early stopping is a heuristic that, while effective, primarily relies on ad hoc parameters and metrics. Optimizing when to stop remains a challenge. In this paper, we suggest that for some biomedical applications, a natural dichotomy of invasive/non-invasive measurements, or more generally proximal vs distal measurements of a biological system can be exploited to provide objective advice on early stopping. We discuss the conditions where invasive measurements of a biological process should provide better predictions than non-invasive measurements, or at best offer parity. Hence, if data from an invasive measurement are available locally, or from the literature, that information can be leveraged to know with high certainty whether a model of non-invasive data is overfitted. We present paired invasive/non-invasive cardiac and coronary artery measurements from two mouse strains, one of which spontaneously develops type 2 diabetes, posed as a classification problem. Examination of the various stopping rules shows that generalization is reduced with more training epochs and commonly applied stopping rules give widely different generalization error estimates. The use of an empirically derived training ceiling is demonstrated to be helpful as added information to leverage early stopping in order to reduce overfitting.
 Computer Science Vision Computer Imaging Computer Systems Organization and Communication Networks Data Structures and Information Theory General Information Systems and Communication Service Original Research Pattern Recognition and Graphics Signal Processing and Multimedia Applications Software Engineering/Programming and Operating Systems

Details

Metrics

30 Record Views
Logo image