Scarf enables a highly memory-efficient analysis of large-scale single-cell genomics data

Area of Science:

• Computational Biology
• Genomics
• Bioinformatics

Background:

• Single-cell genomics experiments are rapidly increasing in scale, generating massive datasets.
• High computational demands limit accessibility of large-scale single-cell data analysis.
• Existing tools often require substantial computational resources, posing a barrier for many researchers.

Purpose of the Study:

• To introduce Scarf, a Python package designed for memory-efficient analysis of large-scale single-cell genomics data.
• To enable researchers to analyze millions of cells on standard hardware, including laptops and single-board computers.
• To provide a modular and interoperable toolkit for advanced single-cell data processing.

Main Methods:

• Development of a modular Python package (Scarf) with memory-efficient algorithms.
• Implementation of graph-based t-stochastic neighbor embedding and hierarchical clustering.
• Integration of reference-anchored mapping and a subsampling algorithm for rare cell population preservation.

Main Results:

• Scarf demonstrates significant memory and compute-time efficiency on large single-cell RNA-Seq and ATAC-Seq datasets.
• Accurate reference-anchored mapping is achieved while maintaining memory efficiency.
• The subsampling algorithm effectively conserves rare cell populations and lineage trajectories.

Conclusions:

• Scarf democratizes large-scale single-cell data analysis by enabling processing on standard computational devices.
• The package offers a comprehensive framework for advanced processing, subsampling, reanalysis, and integration of atlas-scale datasets.
• Scarf empowers researchers with limited computational resources to conduct sophisticated single-cell genomics studies.