词嵌入作为统计估计器
Neil Dey1, Matthew Singer1, Jonathan P Williams2
1Department of Statistics, North Carolina State University.
概括
本研究介绍了词嵌入的统计框架,通过点向互联信息 (PMI) 解释Word2Vec. 一个新的缺失值估计器提供了一个统计学上合理的替代方案,其性能与Word2Vec.ec.相当.
科学领域:
- 自然语言处理自然语言处理.
- 统计学理论统计学理论
- 机器学习 机器学习
背景情况:
- 词嵌入在NLP中至关重要,但缺乏理论理解.
- 目前的评估依赖于经验性表现,而不是严格的属性.
- 正式推断和不确定性量化需要一个理论基础.
研究的目的:
- 提供关于词嵌入的统计理论视角.
- 在正式的统计模型中解释Word2Vec等经典方法.
- 开发一种新的,统计学上可处理的替代现有的词嵌入技术.
主要方法:
- 为文本数据提出了一个基于copula的统计模型.
- 解释了Word2Vec作为理论点向相互信息 (PMI) 的估计器.
- 在之前的工作基础上开发了基于缺失的价值的估计器.
主要成果:
- 证明了Word2Vec与估计理论PMI的联系.
- 建议的缺失值估计器显示了与Word2Vec.ec相似的估计错误.
- 新的估计器的性能优于基于切割的方法.
- 在IMDb情绪分析任务中实现了与Word2Vec可比的性能.
结论:
- 基于的模型为词嵌入提供了理论基础.
- 缺失值估计器提供了一个统计学上可解释和有效的替代方案.
- 这项工作弥合了经验上的成功与词嵌入中的理论理解之间的差距.
相关概念视频
Estimating Population Standard Deviation
3.3K
When the population standard deviation is unknown and the sample size is large, the sample standard deviation s is commonly used as a point estimate of σ. However, it can sometimes under or overestimate the population standard deviation. To overcome this drawback, confidence intervals are determined to estimate population parameters and eliminate any calculation bias accurately. However, this only applies to random samples from normally distributed populations. Knowing the sample mean and...
3.3K
Estimating Population Mean with Unknown Standard Deviation
8.7K
In practice, we rarely know the population standard deviation. In the past, when the sample size was large, this did not present a problem to statisticians. They used the sample standard deviation s as an estimate for σ and proceeded as before to calculate a confidence interval with close enough results. However, statisticians ran into problems when the sample size was small. A small sample size caused inaccuracies in the confidence interval.
William S. Gosset (1876–1937) of the...
William S. Gosset (1876–1937) of the...
8.7K
What are Estimates?
8.0K
It isn't easy to measure a parameter such as the mean height or the mean weight of a population. So, we draw samples from the population and calculate the mean height or mean weight of the individuals in the sample. This sample data acts as a representative measure of the population parameter. These sample statistics are known as estimates.
The estimate for the mean of a sample is denoted by ͞x, whereas the mean of the population is designated as μ. Further, parameters such...
The estimate for the mean of a sample is denoted by ͞x, whereas the mean of the population is designated as μ. Further, parameters such...
8.0K
Estimating Population Mean with Known Standard Deviation
9.6K
To construct a confidence interval for a single unknown population mean μ, where the population standard deviation is known, we need sample mean as an estimate for μ and we need the margin of error. Here, the margin of error (EBM) is called the error bound for a population mean (abbreviated EBM). The sample mean is the point estimate of the unknown population mean μ.
The confidence interval estimate will have the form as follows:
(point estimate - error bound, point estimate +...
The confidence interval estimate will have the form as follows:
(point estimate - error bound, point estimate +...
9.6K
Statistical Significance
21.0K
Once data is collected from both the experimental and the control groups, a statistical analysis is conducted to find out if there are meaningful differences between the two groups. A statistical analysis determines how likely any difference found is due to chance (and thus not meaningful). In psychology, group differences are considered meaningful, or significant, if the odds that these differences occurred by chance alone are 5 percent or less. Stated another way, if we repeated this...
21.0K
Empirical Method to Interpret Standard Deviation
9.3K
The empirical rule, also known as the three-sigma rule, allows a statistician to interpret the standard deviation in a normally distributed dataset. The rule states that 68% of the data lies within one standard deviation from the mean, 95% lies within two standard deviations from the mean, and 99.7% lies within three standard deviations from the mean. Additionally, this rule is also called the 68-95-99.7 rule.
This rule is used widely in statistics to calculate the proportion of data values...
This rule is used widely in statistics to calculate the proportion of data values...
9.3K

