使用三参数项目响应理论确定Angoff方法中的标准制定者内部不一致性
Mohsen Tavakol1, David O'Brien2, Claire Stewart2
1Medical Education Centre, School of Medicine, The University of Nottingham, UK.
International journal of medical education
|September 7, 2023
概括
这项研究使用物品响应理论 (IRT) 来分析医学学生的Angoff评分,发现标准制定者普遍一致. 这种方法有助于减少教育评估中的错误.
科学领域:
- 医学教育 医学教育
- 心理测量 心理测量 心理测量
- 教育评估的教育评估
背景情况:
- 在医学教育中制定标准对于确定通过分数至关重要.
- 安戈夫方法被广泛使用,但可以表现出标准制定者内部的变化.
- 隐性特征理论为理解评估变性提供了一个框架.
研究的目的:
- 为了量化Angoff评级的标准制定者内部变化.
- 评估由不同的标准制定者确定的通过分数的变化.
- 应用项目响应理论 (IRT) 来分析判断标准设定的一致性.
主要方法:
- 一项涉及358名最后一年医学学生的横截面研究.
- 作为总结性评估的一部分进行的两项基于知识的测试.
- 使用三参数物品响应理论 (IRT) 模型分析Angoff评级.
主要成果:
- 三参数IRT模型有效地分析了个别标准制定者的结果.
- 标准制定者在两个测试中显示出整体一致性.
- 平均Angoff评级和条件概率之间的强烈正相关性表明了标准制定者之间的有效性.
结论:
- 该研究验证了IRT用于分析标准制定中的判断不一致性的使用.
- 评估提供者应该采用这种方法来最大限度地减少错误的通过/失败决策.
- 通过先进的心理测量技术,可以实现和测量一致的标准设定.
更多相关视频
14:14The Innovation Arena: A Method for Comparing Innovative Problem-Solving Across Groups
Published on: May 13, 2022
5.9K
09:00Author Spotlight: Validation of SICOLE-R for Assessing Cognitive and Reading Skills in Spanish-Speaking Children and Its Role in Personalized Education
Published on: August 16, 2024
802
相关概念视频
Friedman Two-way Analysis of Variance by Ranks
229
Friedman's Two-Way Analysis of Variance by Ranks is a nonparametric test designed to identify differences across multiple test attempts when traditional assumptions of normality and equal variances do not apply. Unlike conventional ANOVA, which requires normally distributed data with equal variances, Friedman's test is ideal for ordinal or non-normally distributed data, making it particularly useful for analyzing dependent samples, such as matched subjects over time or repeated measures...
229
One-Way ANOVA: Equal Sample Sizes
3.3K
One-Way ANOVA can be performed on three or more samples with equal or unequal sample sizes. When one-way ANOVA is performed on two datasets with samples of equal sizes, it can be easily observed that the computed F statistic is highly sensitive to the sample mean.
Different sample means can result in different values for the variance estimate: variance between samples. This is because the variance between samples is calculated as the product of the sample size and the variance between the...
Different sample means can result in different values for the variance estimate: variance between samples. This is because the variance between samples is calculated as the product of the sample size and the variance between the...
3.3K
One-Way ANOVA: Unequal Sample Sizes
5.8K
One-way ANOVA can be performed on three or more samples of unequal sizes. However, calculations get complicated when sample sizes are not always the same. So, while performing ANOVA with unequal samples size, the following equation is used:
5.8K
Bonferroni Test
2.8K
The Bonferroni test is a statistical test named after Carlo Emilio Bonferroni, an Italian mathematician best known for Bonferroni inequalities. This statistical test is a type of multiple comparison test to determine which means are different than the rest. Bonferroni test can minimize the Type 1 error by reducing the significance level alpha, which otherwise increases with sample pairs.
The means of different samples are first paired in all possible combinations.
The null hypothesis of the...
The means of different samples are first paired in all possible combinations.
The null hypothesis of the...
2.8K
Testing a Claim about Standard Deviation
2.5K
A complete procedure to test a claim about population standard deviation or population variance is explained here.
The hypothesis testing for the claim of population standard deviation (or variance) requires the data and samples to be random and unbiased. The population distribution also must be normal. There is no specific requirement on the sample size as the estimation is based on the chi-square distribution.
As a first step, the hypothesis (null and alternative) concerning the claim about...
The hypothesis testing for the claim of population standard deviation (or variance) requires the data and samples to be random and unbiased. The population distribution also must be normal. There is no specific requirement on the sample size as the estimation is based on the chi-square distribution.
As a first step, the hypothesis (null and alternative) concerning the claim about...
2.5K
One-Way ANOVA
7.9K
One-way ANOVA analyzes more than three samples categorized by one factor. For example, it can compare the average mileage of sports bikes. Here, the data is categorized by one factor - the company. However, one-way ANOVA cannot be used to simultaneously compare the sample mean of three or more samples categorized by two factors. An example of two factors would be sports bikes from different companies driven in different terrains, such as a desert or snowy landscape. Here, two-way ANOVA is used...
7.9K
