Recursive Clustering of Cellular Diversity in scRNA-Seq Data

In scRNA-seq analysis, cell clusters are typically defined by a single round of feature extraction and clustering. This approach may miss phenotypic differences in cell types that are characterized by genes not sufficiently represented in the feature set derived using all cells, such as rare cell types. This work explores an alternative approach, where cell clusters are identified by recursively performing feature extraction and clustering on previously identified clusters, such that each subclustering step uses features that are more specific to distinguishing the higher-resolution subclusters. We benchmark this recursive approach against the conventional, nonrecursive clustering approach and demonstrate that the recursive method results in robust improvement in cell type detection on four scRNA-seq datasets across a wide range of clustering resolution parameters. We apply the recursive approach to cluster scRNA-seq data obtained from patients with Crohn's disease belonging to three clinical phenotypes and observe that recursive clustering captures phenotypic differences only visible at specific levels of granularity within an interpretable hierarchical framework while defining cell clusters within a gene expression feature space more specific to each cluster.