Empirical Evaluation of Single-Cell Foundation Models for Predicting Cancer Outcomes.

Area of Science:

• Computational biology
• Genomics
• Translational oncology

Background:

• Foundation models pretrained on large-scale single-cell RNA sequencing (scRNA-seq) data offer potential for translational cancer research.
• The application of these models in patient-level clinical settings is not yet fully understood.

Purpose of the Study:

• To systematically evaluate the performance of emerging single-cell foundation models (scFMs) on cancer-specific tasks.
• To compare scFMs against baseline approaches in various training conditions for clinical applications.

Main Methods:

• Evaluated nine scFMs and three baseline models across six cancer tasks (e.g., subtype classification, treatment response prediction).
• Conducted 1,170 supervised and 130 unsupervised experiments under zero-shot, continual training, and fine-tuning conditions.
• Assessed model utility in patient-level single-cell analysis.

Main Results:

• Current scFMs demonstrated strong performance in specific tasks like tumor microenvironment cell annotation.
• scFMs showed limited advantages over simpler baseline models in predicting clinical and biological outcomes for cancer patients.
• Performance varied across different tasks and evaluation conditions.

Conclusions:

• Rigorous evaluation on biologically and clinically relevant tasks is crucial for the responsible application of scFMs in precision oncology.
• Further methodological innovation and larger cancer scRNA-seq datasets are necessary for advancing scFMs.
• Current scFMs may not yet fully capture the complexity required for predicting patient-level outcomes in cancer.