Related Experiment Video
Updated: Jul 22, 2025
03:14
Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
625
Quality indices for topic model selection and evaluation: a literature review and case study
Christopher Meaney1, Therese A Stukel2, Peter C Austin2
1Department of Family and Community Medicine, University of Toronto, 500 University Ave, Toronto, ON, M5G1V7, Canada. christopher.meaney@utoronto.ca.
BMC Medical Informatics and Decision Making
|July 22, 2023
Summary
Evaluating unsupervised topic models requires careful consideration of various quality metrics. Different metrics favor models of varying complexity, and human judgment is crucial for optimal model selection in clinical text analysis.
Area of Science:
- Machine Learning
- Natural Language Processing
- Computational Linguistics
Background:
- Topic models are unsupervised machine learning tools for analyzing large document collections.
- Assessing the quality of topic models is crucial for effective summarization and retrieval.
- Non-negative matrix factorization (NMF) is a common technique for estimating topic models.
Conclusions:
- Topic model quality indices are valuable for guiding model selection but may not align with human judgment.
- Different metrics capture distinct aspects of model performance.
- A combination of quantitative metrics and human validation is recommended for robust topic model appraisal.
Related Concept Videos
01:16Goodness-of-Fit Test
3.5K
The goodness-of-fit test is a type of hypothesis test which determines whether the data "fits" a particular distribution. For example, one may suspect that some anonymous data may fit a binomial distribution. A chi-square test (meaning the distribution for the hypothesis test is chi-square) can be used to determine if there is a fit. The null and alternative hypotheses may be written in sentences or stated as equations or inequalities. The test statistic for a goodness-of-fit test is given as...
3.5K
01:19Expected Frequencies in Goodness-of-Fit Tests
2.6K
A goodness-of-fit test is conducted to determine whether the observed frequency values are statistically similar to the frequencies expected for the dataset. Suppose the expected frequencies for a dataset are equal such as when predicting the frequency of any number appearing when casting a die. In that case, the expected frequency is the ratio of the total number of observations (n) to the number of categories (k).
2.6K
01:02Quantifying and Rejecting Outliers: The Grubbs Test
1.7K
Sometimes, a data set can have a recorded numerical observation that greatly deviates from the rest of the data. Assuming that the data is normally distributed, a statistical method called the Grubbs test can be used to determine whether the observation is truly an outlier. To perform a two-tailed Grubbs test, first, calculate the absolute difference between the outlier and the mean. Then, calculate the ratio between this difference and the standard deviation of the sample. This...
1.7K
02:16Measures of Central Tendency
16.0K
The "center" of a data set is also a way of describing location. The two most widely used measures of the "center" of the data are the mean (average) and the median. The words "mean" and "average" are often used interchangeably. The substitution of one word for the other is common practice. The technical term is "arithmetic mean" and "average" is technically a center location. However, in practice among non-statisticians,...
16.0K
01:13Accuracy and Errors in Hypothesis Testing
222
Hypothesis testing is a fundamental statistical tool that begins with the assumption that the null hypothesis H0 is true. During this process, two types of errors can occur: Type I and Type II. A Type I error refers to the incorrect rejection of a true null hypothesis, while a Type II error involves the failure to reject a false null hypothesis.
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5%...
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5%...
222
01:52Accuracy and Precision
9.0K
Scientists typically make repeated measurements of a quantity to ensure the quality of their findings and to evaluate both the precision and the accuracy of their results. Measurements are said to be precise if they yield very similar results when repeated in the same manner. A measurement is considered accurate if it yields a result that is very close to the true or the accepted value. Precise values agree with each other; accurate values agree with a true value. Highly accurate...
9.0K