Transformers in single-cell omics: a review and new perspectives

Area of Science:

• Computational Biology
• Genomics
• Machine Learning

Background:

• Single-cell omics data is rapidly expanding, offering rich insights into cellular properties.
• Current single-cell models struggle to fully leverage the complexity of these large-scale datasets.
• Transformers excel in other fields for handling heterogeneous, large-scale data.

Purpose of the Study:

• To explore the potential of transformer models in single-cell biology.
• To review existing applications of transformers in single-cell analysis.
• To identify limitations and future research directions for machine learning in single-cell biology.

Main Methods:

• Review of transformer architecture and its adaptations for single-cell data.
• Comprehensive survey of current transformer applications in single-cell analysis.
• Critical discussion of technical challenges and future potential.

Main Results:

• Transformers offer a powerful architecture for generalizing across heterogeneous, large-scale single-cell datasets.
• Existing applications demonstrate the utility of transformers in various single-cell analysis tasks.
• Identified limitations and challenges provide a roadmap for future development.

Conclusions:

• Transformers have the potential to revolutionize single-cell modeling, similar to their impact in other domains.
• Further research is needed to overcome technical challenges and fully realize the potential of transformers in single-cell biology.
• This review provides a structured outlook for integrating machine learning and single-cell biology.