Self-supervised deep clustering of single-cell RNA-seq data to hierarchically detect rare cell populations

Area of Science:

• Computational Biology
• Genomics
• Bioinformatics

Background:

• Single-cell RNA sequencing (scRNA-seq) enables gene expression analysis at the individual cell level.
• Clustering is crucial for scRNA-seq data analysis but faces challenges due to data sparsity and high dimensionality.
• Existing deep learning clustering methods often fail to capture data distributions or cell relationships effectively.

Purpose of the Study:

• To develop a novel deep learning-based hierarchical clustering tool for scRNA-seq data.
• To improve the accuracy and efficiency of cell clustering, especially for detecting rare cell populations.
• To address limitations of existing methods in handling large-scale, high-dimensional scRNA-seq datasets.

Main Methods:

• Introduction of DeepScena, a hierarchical clustering tool for scRNA-seq data.
• Incorporation of nonlinear dimension reduction techniques.
• Utilized a negative binomial-based convolutional autoencoder for data fitting and a self-supervision model for cell similarity enhancement.

Main Results:

• DeepScena demonstrated superior clustering accuracy compared to seven popular tools across multiple large-scale scRNA-seq datasets.
• The tool showed high proficiency in identifying rare cell populations within large, complex datasets.
• Application to multiple myeloma scRNA-seq data successfully identified known cell types and novel subpopulations in monocytes, T cells, and NK cells.

Conclusions:

• DeepScena offers a significant advancement in scRNA-seq data analysis, particularly for clustering and rare cell population identification.
• The method's ability to handle large datasets and complex data structures makes it a valuable tool for biological research.
• DeepScena provides a robust framework for uncovering cellular heterogeneity and improving downstream analyses in single-cell genomics.