自动项目生成:项目变体对性能和标准设置的影响
R Westacott1, K Badger2, D Kluth3
1Birmingham Medical School, University of Birmingham, Birmingham, UK. r.j.westacott@bham.ac.uk.
BMC medical education
|September 11, 2023
概括
自动项目生成 (AIG) 创建了各种各样的测试项目,但这些变体在学生表现上显示出比标准设置更大的差异. 需要进一步的研究来了解医疗教育评估中的这些绩效差异.
科学领域:
- 医学教育 医学教育
- 评估科学 评估科学
背景情况:
- 自动项目生成 (AIG) 软件从单个问题模型创建多个测试项目.
- 关于项目变体是否影响学生表现或标准设置的数据有限.
- 本研究使用AIG生成的测试来调查性能和标准设置差异.
研究的目的:
- 使用AIG生成的四个不同的测试之间的学生表现和标准设定数据进行比较.
- 评估异形项目变体对医学学生评估结果的影响.
- 评估AIG在创建可比性评估工具方面的可靠性和有效性.
主要方法:
- 从50个MCQ模型中使用AIG软件生成了四个同型50项MCQ测试.
- 作为在线形成性评估对英国医学学生的最后一年进行的测试.
- 应用了修改后的Angoff方法来制定标准,并对项目变体进行了专题分析.
主要成果:
- 来自12所英国医学院的2218名学生参加了这次活动.
- 平均纸张设施在0.55-0.61之间;切割分数在0.58-0.61.1之间.
- 观察到的显著差异:20个项目模型的设施差异>0.15,10个标准设置差异>0.1.1.
结论:
- 项目设施表现出比标准设置得分更大的变化.
- 绩效差异可能源于在新手学习者中破坏临床推理的变体.
- 学校一级的绩效差异需要进一步调查,以充分解释观察到的差异.
相关概念视频
Testing a Claim about Standard Deviation
2.5K
A complete procedure to test a claim about population standard deviation or population variance is explained here.
The hypothesis testing for the claim of population standard deviation (or variance) requires the data and samples to be random and unbiased. The population distribution also must be normal. There is no specific requirement on the sample size as the estimation is based on the chi-square distribution.
As a first step, the hypothesis (null and alternative) concerning the claim about...
The hypothesis testing for the claim of population standard deviation (or variance) requires the data and samples to be random and unbiased. The population distribution also must be normal. There is no specific requirement on the sample size as the estimation is based on the chi-square distribution.
As a first step, the hypothesis (null and alternative) concerning the claim about...
2.5K
Variability: Analysis
158
Measures of variability are statistical metrics that reveal the dispersion pattern within a dataset. They are pivotal in biostatistics, providing insights into the heterogeneity within health and biological data. Variability signifies the degree to which data points diverge from one another, helping researchers understand the potential range of values and associated uncertainty within the data.
The range is a simple measure of variability, indicating the difference between the highest and...
The range is a simple measure of variability, indicating the difference between the highest and...
158
What is Variation?
11.9K
Apart from the measures of central tendency, distribution, outliers, and the changing characteristics of data with time, an important characteristic of any data set is its variation or spread. In some data sets, the data values are concentrated closely near the mean; in others, the data values are more widely spread out from the mean.
The range, standard deviation, standard error, and variance are the different measures of variation.
Range: The range is the difference between its maximum and...
The range, standard deviation, standard error, and variance are the different measures of variation.
Range: The range is the difference between its maximum and...
11.9K
Variation: Normal Distribution, Range, and Standard Deviation
22.3K
In the field of psychology, there are several ways to organize measurements of a trait, feature, or characteristic (i.e., variables). Qualitative data, such as ethnicity, can be tabulated into a frequency count to provide information about the proportion, as well as the variety of groups in a sample or population. On the other hand, researchers can perform a wider set of calculations on quantitative data. The mean, mode, and median, for instance, are central tendency measures to identify a...
22.3K
Factorial Design
13.1K
Factorial Analysis is an experimental design that applies Analysis of Variance (ANOVA) statistical procedures to examine a change in a dependent variable due to more than one independent variable, also known as factors. Changes in worker productivity can be reasoned, for example, to be influenced by salary and other conditions, such as skill level. One way to test this hypothesis is by categorizing salary into three levels (low, moderate, and high) and skills sets into two levels (entry level...
13.1K
Randomized Experiments
7.0K
The randomization process involves assigning study participants randomly to experimental or control groups based on their probability of being equally assigned. Randomization is meant to eliminate selection bias and balance known and unknown confounding factors so that the control group is similar to the treatment group as much as possible. A computer program and a random number generator can be used to assign participants to groups in a way that minimizes bias.
Simple randomization
Simple...
Simple randomization
Simple...
7.0K


