一个实验设计来调查项目参数漂移的实验设计
Peter Baldwin1, Irina Grabovsky1, Kimberly A Swygert1
1NBME, Philadelphia, PA, USA.
Applied psychological measurement
|January 27, 2025
概括
一个新的策略使用未暴露的项目来检测项目参数漂移,这是测试中的一个常见问题. 虽然这表明在执照考试中存在偏移,但在0.05级别,结果在统计学上并不显著.
科学领域:
- 心理测量 心理测量 心理测量
- 教育测量教育的测量
- 统计建模 统计建模
背景情况:
- 当所有项目都暴露时,检测项目参数漂移的传统方法是不够的.
- 项目参数漂移可以显著影响测试的有效性和公平性.
- 高风险考试需要强大的方法来确保得分的完整性.
研究的目的:
- 提出和说明一种用于检测物品参数漂移的新策略,仅使用未暴露的物品.
- 解决现有漂移检测方法在完全暴露物品场景中的局限性.
- 在现实世界高风险测试环境中评估拟议方法的有效性.
主要方法:
- 采用分层随机采样方法,在实验设计中部署未暴露的项目.
- 该策略侧重于分析未暴露项目的子集数据,以推断暴露项目的潜在漂移.
- 该方法用于调查在许可证考试中观察到的意想不到的得分增加.
主要成果:
- 将拟议方法应用于许可证考试数据的应用产生了表明项目参数漂移的结果.
- 观察到的得分增加表明项目特征的潜在变化.
- 在常规的0.05α水平上没有达到统计显著性,这表明结果不确切.
结论:
- 拟议的策略提供了一种可行的方法来检测项目参数漂移,特别是当所有项目都面临风险时.
- 需要进一步的研究和验证来完善该方法并确认其灵敏度.
- 这些发现强调了在标准化测试中持续监测项目参数偏移的重要性.
相关概念视频
Experimental Designs
11.1K
An experimental design is a systematic process that allows researchers to evaluate the relationship between dependent and independent variables. There are three widely used types of experimental design - pre-experimental design, true experimental design, and quasi-experimental design. In pre-experimental design, the researcher compares the data before and after some interventions or treatments. The true-experimental design has more than one purposefully created group, a commonly measured...
11.1K
Factorial Design
13.0K
Factorial Analysis is an experimental design that applies Analysis of Variance (ANOVA) statistical procedures to examine a change in a dependent variable due to more than one independent variable, also known as factors. Changes in worker productivity can be reasoned, for example, to be influenced by salary and other conditions, such as skill level. One way to test this hypothesis is by categorizing salary into three levels (low, moderate, and high) and skills sets into two levels (entry level...
13.0K
Testing a Claim about Standard Deviation
2.4K
A complete procedure to test a claim about population standard deviation or population variance is explained here.
The hypothesis testing for the claim of population standard deviation (or variance) requires the data and samples to be random and unbiased. The population distribution also must be normal. There is no specific requirement on the sample size as the estimation is based on the chi-square distribution.
As a first step, the hypothesis (null and alternative) concerning the claim about...
The hypothesis testing for the claim of population standard deviation (or variance) requires the data and samples to be random and unbiased. The population distribution also must be normal. There is no specific requirement on the sample size as the estimation is based on the chi-square distribution.
As a first step, the hypothesis (null and alternative) concerning the claim about...
2.4K
Crossover Experiments
2.7K
Crossover experiments, also called the repeated-measurements design, is a study design in which all experimental units are exposed to all treatments in different periods. Crossover experiments are generally used in psychology, the pharmaceutical industry, agriculture, and medicine.
Crossover designs are performed even with smaller sample sizes since the samples can act as their controls. These are better than simple randomized trials since patients are exposed to all the treatments.
Crossover designs are performed even with smaller sample sizes since the samples can act as their controls. These are better than simple randomized trials since patients are exposed to all the treatments.
2.7K
Group Design
8.9K
The most basic experimental design involves two groups: the experimental group and the control group. The two groups are designed to be the same except for one difference— experimental manipulation. The experimental group gets the experimental manipulation—that is, the treatment or variable being tested—and the control group does not. Since experimental manipulation is the only difference between the experimental and control groups, we can be sure that any differences between...
8.9K
Longitudinal Research
11.8K
Sometimes we want to see how people change over time, as in studies of human development and lifespan. When we test the same group of individuals repeatedly over an extended period of time, we are conducting longitudinal research. Longitudinal research is a research design in which data-gathering is administered repeatedly over an extended period of time. For example, we may survey a group of individuals about their dietary habits at age 20, retest them a decade later at age 30, and then again...
11.8K


