Selective Inference for Hierarchical Clustering

Area of Science:

• Statistical methodology
• Bioinformatics
• Data science

Background:

• Classical statistical tests assume groups are defined a priori.
• Clustering-based group definitions lead to inflated Type I error rates with traditional tests.
• This issue persists even with independent datasets for clustering and testing.

Purpose of the Study:

• To develop a selective inference approach for testing mean differences between two clusters.
• To control the Type I error rate when hypotheses are data-driven.
• To provide an efficient method for computing exact p-values for hierarchical clustering.

Main Methods:

• Proposed a selective inference procedure to address inflated Type I errors.
• Developed efficient computation of exact p-values for agglomerative hierarchical clustering.
• Validated the method using simulated and single-cell RNA-sequencing data.

Main Results:

• The proposed selective inference method effectively controls the Type I error rate.
• Demonstrated accurate p-value computation for data-driven cluster comparisons.
• Successfully applied the method to real-world single-cell RNA-sequencing data.

Conclusions:

• Selective inference is crucial for valid hypothesis testing after data-driven clustering.
• The developed method offers a statistically sound approach for comparing means between clusters.
• This work has implications for fields utilizing clustering, such as single-cell genomics.