RNA2seg: a generalist model for cell segmentation in image-based spatial transcriptomics

Area of Science:

• Genomics
• Bioinformatics
• Cell Biology

Background:

• Imaging-based spatial transcriptomics provides high-resolution RNA mapping.
• Accurate cell segmentation is a critical bottleneck for assigning RNA molecules to specific cells.

Purpose of the Study:

• To develop a novel, accurate cell segmentation algorithm for imaging-based spatial transcriptomics.
• To improve the assignment of RNA molecules to individual cells in spatial transcriptomic data.

Main Methods:

• Developed RNA2seg, a segmentation algorithm utilizing a teacher-student training scheme.
• Trained RNA2seg on over 4 million cells from MERFISH and CosMx datasets across seven organs.
• Integrated RNA point clouds with membrane and nuclear staining data.

Main Results:

• RNA2seg demonstrates superior performance in cell segmentation.
• Achieved high accuracy in zero-shot and few-shot segmentation settings.
• Validated performance on manually annotated datasets.

Conclusions:

• RNA2seg offers a significant advancement in cell segmentation for spatial transcriptomics.
• The algorithm enhances the precision of RNA molecule localization within cells.
• RNA2seg shows robust performance across diverse datasets and conditions.