Deep learning in biology faces a transferability crisis.

Area of Science:

• Deep learning
• Computational biosciences
• Model generalizability

Background:

• Generalizability is a core goal in deep learning research.
• Misleading claims of model transferability can compromise scientific integrity and reproducibility.
• Accurate performance evaluation is essential for advancing biosciences.

Purpose of the Study:

• To highlight the problem of misleading transferability claims in deep learning within the biosciences.
• To underscore the negative impact of these claims on reliable performance evaluation.
• To propose actionable solutions for improving model generalizability assessment.

Main Methods:

• Literature review on deep learning generalizability in biosciences.
• Analysis of case studies demonstrating transferability issues.
• Development of a framework for evaluating model generalizability.

Main Results:

• Identified specific instances where transferability claims were unsubstantiated.
• Quantified the potential negative impact on research reproducibility.
• Proposed criteria for robust evaluation of model generalizability.

Conclusions:

• Addressing misleading transferability claims is crucial for scientific progress in the biosciences.
• Implementing standardized evaluation protocols will enhance the reliability of deep learning models.
• Future research should focus on developing more robust methods for assessing model generalizability.