Consequences of training data composition for deep learning models in single-cell biology.

Area of Science:

• Computational biology
• Genomics
• Machine learning

Background:

• Foundation models offer potential for single-cell transcriptomics analyses, especially with sparse data.
• Current single-cell foundation models train on large corpora, often overlooking training data composition's impact on performance.
• Large language model research highlights the critical role of training data composition in shaping model behavior.

Purpose of the Study:

• To systematically investigate how training dataset composition affects deep learning models for single-cell transcriptomics.
• To evaluate model generalization to unseen cell types and disease states.
• To identify strategies for optimizing future single-cell foundation models.

Main Methods:

• Focused on human hematopoiesis as a model system.
• Included diverse cell types from adult and developing tissues, disease states, and perturbation atlases.
• Systematically varied training dataset composition to assess model behavior.

Main Results:

• Models demonstrated poor generalization to unseen cell types.
• Incorporating malignant cells into training data did not consistently improve modeling of unseen malignant cells.
• Including embryonic stem cell differentiation data enhanced performance on out-of-distribution tasks.

Conclusions:

• Training data diversity is crucial for effective single-cell foundation models.
• Current training strategies may lead to poor generalization.
• Future models should prioritize curated, diverse datasets for improved performance and broader applicability.