白化揭示了一个计数矩阵的等级
Boris Landa1, Thomas T C K Zhang2, Yuval Kluger1
1Program in Applied Mathematics, Yale University.
SIAM journal on mathematics of data science
|August 14, 2023
概括
我们介绍了双白化,这是一种用于估计计数数据矩阵的排名的新方法,对于主要组件分析 (PCA) 至关重要. 这种技术有效地处理Poisson和相关计数数据中的异构分级噪声,提高排名估计的准确性.
科学领域:
- 数据科学数据科学数据科学
- 统计学学习 统计学学习
- 矩阵分析是一门学科.
背景情况:
- 估计矩阵排名对于主要组件分析 (PCA) 和数据分析至关重要.
- 传统方法在计数数据矩阵 (例如,Poisson) 上因异构杂而扎.
- 随机矩阵理论提供了工具,但需要适应计数数据.
研究的目的:
- 开发一种可靠的方法来估计具有未知方差的波桑随机矩阵的等级.
- 为了应对对排位估计计计数数据中异构排列噪声的挑战.
- 为各种计数分布和缺失数据提供一种可概括的方法.
主要方法:
- 提出了一种双白化程序来缩放数据矩阵.
- 保证的噪声频谱符合马尔琴科-帕斯图尔 (MP) 关于等级选择的法律.
- 利用Sinkhorn-Knopp算法从观察中估计缩放因子.
主要成果:
- 在没有事先知识的情况下,Biwhitening有效地估计了Poisson参数矩阵的排名.
- 该方法扩展到具有二次平均变量关系的其他分布 (例如二项式,玛).
- 数字实验和现实世界数据集 (scRNA-seq,Hi-C,主题建模) 验证了该方法.
结论:
- Biwhitening提供了一种强大而通用的工具,用于在损坏的计数数据矩阵中进行排名估计.
- 该方法在具有挑战性的场景和各种应用中表现出卓越的性能.
- 这项工作在各种科学领域推进了基于计数的数据分析.
相关概念视频
01:02Ranks
261
Unlike parametric methods, nonparametric statistics are ideal for nominal and ordinal data, requiring fewer assumptions about the population's nature or distribution. This makes nonparametric methods easier to apply and interpret, as they do not depend on parameters like mean or standard deviation. One common approach in nonparametric analysis is to sort data according to a specific criterion. For instance, we might arrange weather data from hottest to coldest days in a month or rank cities...
261
01:21Wilcoxon Rank-Sum Test
227
The Wilcoxon rank-sum test, also known as the Mann-Whitney U test, is a nonparametric test used to determine if there is a significant difference between the distributions of two independent samples. This test is designed specifically for two independent populations and has the following key requirements:
227
01:21Friedman Two-way Analysis of Variance by Ranks
235
Friedman's Two-Way Analysis of Variance by Ranks is a nonparametric test designed to identify differences across multiple test attempts when traditional assumptions of normality and equal variances do not apply. Unlike conventional ANOVA, which requires normally distributed data with equal variances, Friedman's test is ideal for ordinal or non-normally distributed data, making it particularly useful for analyzing dependent samples, such as matched subjects over time or repeated measures...
235
01:14Wilcoxon Signed-Ranks Test for Median of Single Population
167
The Wilcoxon signed-rank test for the median of a single population is a nonparametric test used to evaluate whether the median of a population differs from a specified value. Unlike parametric tests, it does not require data to follow a normal distribution, making it suitable for non-normal or small samples. The test begins by calculating the difference (d) between each observation and the hypothesized median. The absolute values of these differences are ranked in ascending order, with ties...
167
01:09Wilcoxon Signed-Ranks Test for Matched Pairs
164
The Wilcoxon signed-rank test for matched pairs evaluates the null hypothesis by combining the ranks of differences with their signs. It essentially tests whether the median of the differences in a population of matched pairs is zero. Since the test incorporates more information than the sign test, it generally yields more trustable conclusions. This test also does not require the data to follow a normal distribution, but two conditions must be met for it to be applicable: (1) the data must...
164
01:16Kendall's Tau Test
727
Kendall's tau test, also known as the Kendall rank coefficient test, is a nonparametric method for assessing association between two variables. This test is particularly useful for identifying significant correlations when the distributions of the sample and population are unknown. Developed in 1938 by the British statistician Sir Maurice George Kendall, the tau coefficient (denoted as τ) serves as a rank correlation coefficient, with values ranging from -1 to +1.
A τ value...
A τ value...
727