Invasive or More Direct Measurements Can Provide an Objective Early-Stopping Ceiling for Training Deep Neural...

Area of Science:

• Biomedical Engineering
• Machine Learning in Medicine
• Computational Biology

Background:

• Early stopping is a common heuristic to prevent model overfitting and improve generalization.
• Current early stopping methods rely on ad hoc parameters and metrics, making optimization challenging.
• Objective criteria for early stopping are needed, especially in biomedical applications.

Purpose of the Study:

• To propose a novel approach for early stopping in biomedical applications using invasive/non-invasive measurement dichotomies.
• To investigate the utility of invasive measurements for validating non-invasive data models and detecting overfitting.
• To provide objective advice on optimizing early stopping in machine learning models for biological systems.

Main Methods:

• Exploiting the dichotomy between proximal (invasive) and distal (non-invasive) measurements of biological systems.
• Utilizing paired invasive/non-invasive cardiac and coronary artery measurements from mouse models.
• Analyzing classification performance and generalization errors across different training epochs and stopping rules.

Main Results:

• Invasive measurements can provide objective criteria to assess overfitting in models trained on non-invasive data.
• Commonly used early stopping rules yield varied estimates of generalization error.
• An empirically derived training ceiling aids in leveraging early stopping to reduce overfitting.

Conclusions:

• The invasive/non-invasive measurement dichotomy offers a robust strategy for objective early stopping in biomedical machine learning.
• Leveraging invasive data can significantly improve the reliability of early stopping decisions, reducing overfitting.
• This method enhances model generalization for novel data in complex biological systems.