Crafted experiments to evaluate feature selection methods for single-cell RNA-seq data.

Area of Science:

• Computational Biology
• Bioinformatics
• Genomics

Background:

• Analyzing single-cell RNA sequencing (scRNA-seq) data involves numerous computational methods.
• Benchmarking these analysis methods is challenging due to the lack of ground truth datasets.
• Existing methods struggle to rigorously evaluate gene selection and clustering algorithms.

Purpose of the Study:

• To introduce a novel approach called 'crafted experiments' for evaluating scRNA-seq analysis methods.
• To assess the performance of a new suite of univariate distribution-oriented feature selection methods, GOF.
• To provide a framework for robustly comparing gene selection and clustering techniques in scRNA-seq.

Main Methods:

• Developed 'crafted experiments' by perturbing signals within real scRNA-seq datasets.
• Proposed and evaluated the GOF (Goodness-of-Fit) suite of feature selection methods.
• Utilized varying crafting strategies to determine optimal contexts for each GOF method.

Main Results:

• Crafted experiments effectively enabled the evaluation of feature selection methods.
• The GOF methods demonstrated robustness in identifying crafted features.
• GOF methods performed well on real, non-crafted scRNA-seq datasets, showing context-specific advantages.

Conclusions:

• Crafted experiments offer a viable solution for benchmarking scRNA-seq analysis tools.
• The GOF package provides effective univariate feature selection for scRNA-seq data.
• The study provides open-source tools for constructing crafted experiments and evaluating feature selection methods.