评估基于调查数据估计的二进制结果分类器
Adway S Wadekar1, Jerome P Reiter
1From the Department of Statistical Science, Duke University, Durham, NC.
Epidemiology (Cambridge, Mass.)
|August 14, 2024
概括
使用调查权重可以改善对复杂调查数据的预测模型评估. 权重指标准确地反映了人口表现,与未加权的指标不同,特别是在减轻阶级不平衡的情况下.
科学领域:
- 流行病学 流行病学
- 健康科学 卫生科学 卫生科学
- 社会和行为科学 社会和行为科学
背景情况:
- 调查是重要的研究工具,但通常使用复杂的抽样设计,而不是简单的随机抽样.
- 调查受访者通常被分配权重,以考虑到不平等的选择概率.
- 在调查数据上评估预测模型需要仔细考虑这些复杂的设计.
研究的目的:
- 证明使用调查权重来评估预测模型质量的好处.
- 在复杂的调查数据上比较加权与未加权的绩效指标.
- 评估权重对训练有素模型的影响,以缓解类失衡.
主要方法:
- 描述模型评估统计数据 (例如,灵敏度,特异性) 作为有限的种群数量.
- 使用原始调查数据的随机子集进行测试的计算调查加权估计.
- 通过使用国家药物使用和健康调查和国家并发症调查数据进行模拟.
主要成果:
- 使用样本测试数据的未加权指标可能不准确地代表了人口的表现.
- 权重指标适当调整复杂的抽样设计,提供准确的人口估计.
- 权重指标的好处仍然存在,即使模型是通过对阶级不平衡的上抽样进行训练.
结论:
- 调查权重对于对复杂的调查数据进行准确的预测模型性能评估至关重要.
- 权重指标提供了一个更可靠的评估模型对目标人群的概括性.
- 研究人员在评估在复杂调查数据集上训练或测试的模型时,应采用加权指标.
相关概念视频
Testing a Claim about Population Proportion
3.3K
A complete procedure for testing a claim about a population proportion is provided here.
There are two methods of testing a claim about a population proportion: (1) Using the sample proportion from the data where a binomial distribution is approximated to the normal distribution and (2) Using the binomial probabilities calculated from the data.
The first method uses normal distribution as an approximation to the binomial distribution. The requirements are as follows: sample size is large...
There are two methods of testing a claim about a population proportion: (1) Using the sample proportion from the data where a binomial distribution is approximated to the normal distribution and (2) Using the binomial probabilities calculated from the data.
The first method uses normal distribution as an approximation to the binomial distribution. The requirements are as follows: sample size is large...
3.3K
Receiver Operating Characteristic Plot
106
A ROC (Receiver Operating Characteristic) plot is a graphical tool used to assess the performance of a binary classification model by illustrating the trade-off between sensitivity (true positive rate) and specificity (false positive rate). By plotting sensitivity against 1 - specificity across various threshold settings, the ROC curve shows how well the model distinguishes between classes, with a curve closer to the top-left corner indicating a more accurate model. The area under the ROC curve...
106
Expected Frequencies in Goodness-of-Fit Tests
2.5K
A goodness-of-fit test is conducted to determine whether the observed frequency values are statistically similar to the frequencies expected for the dataset. Suppose the expected frequencies for a dataset are equal such as when predicting the frequency of any number appearing when casting a die. In that case, the expected frequency is the ratio of the total number of observations (n) to the number of categories (k).
2.5K
Classification of Signals
424
In signal processing, signals are classified based on various characteristics: continuous-time versus discrete-time, periodic versus aperiodic, analog versus digital, and causal versus noncausal. Each category highlights distinct properties crucial for understanding and manipulating signals.
A continuous-time signal holds a value at every instant in time, representing information seamlessly. In contrast, a discrete-time signal holds values only at specific moments, often denoted as x(n), where...
A continuous-time signal holds a value at every instant in time, representing information seamlessly. In contrast, a discrete-time signal holds values only at specific moments, often denoted as x(n), where...
424
Comparing the Survival Analysis of Two or More Groups
164
Survival analysis is a cornerstone of medical research, used to evaluate the time until an event of interest occurs, such as death, disease recurrence, or recovery. Unlike standard statistical methods, survival analysis is particularly adept at handling censored data—instances where the event has not occurred for some participants by the end of the study or remains unobserved. To address these unique challenges, specialized techniques like the Kaplan-Meier estimator, log-rank test, and...
164
Sensitivity, Specificity, and Predicted Value
228
In healthcare diagnostics, laboratory tests play a crucial role in identifying and diagnosing a wide range of medical conditions. However, interpreting test results is not always straightforward. An abnormal test result does not always confirm the presence of a disease, just as a normal result does not guarantee its absence. To assess the reliability of these diagnostic tools, healthcare practitioners rely on two key statistical indicators: sensitivity and specificity.
Sensitivity is the...
Sensitivity is the...
228


