Related Experiment Video
Updated: Jun 30, 2025
14:14
The Innovation Arena: A Method for Comparing Innovative Problem-Solving Across Groups
Published on: May 13, 2022
5.9K
Standard setting anchor statements: a double cross-over trial of two different methods.
Steven Burr1, Theresa Martin1, James Edwards1
1Peninsula Medical School.
Mededpublish (2016)
|March 15, 2024
Summary
A new standard-setting method focuses on knowledge importance, not item difficulty. This approach yields higher cut-scores and increased judge confidence compared to the Angoff method.
Area of Science:
- Educational Measurement
- Psychometrics
Background:
- The Angoff method is a widely used standard-setting technique.
- This method's philosophical acceptability is questioned.
- An alternative standard-setting approach is proposed.
Purpose of the Study:
- To evaluate an alternative standard-setting method.
- To compare this new method directly with the Angoff method.
- To assess the practicalities and outcomes of the alternative method.
Main Methods:
- A prospective cross-over design was employed.
- Two groups of judges (n=7 and n=8) set standards for two 100-item multiple-choice tests.
- Both the Angoff method and the alternative method were used by each group.
Main Results:
- Both methods required similar completion times.
- The alternative method resulted in a 12-14% higher cut-score.
- The alternative method showed a 5% greater variability in judges' cut-scores and a 3% increase in judge confidence.
Conclusions:
- A novel standard-setting approach was introduced.
- The alternative method offers qualitative advantages over the Angoff method.
- Quantitative differences between the methods were observed to be slight.
Related Concept Videos
01:16Crossover Experiments
2.8K
Crossover experiments, also called the repeated-measurements design, is a study design in which all experimental units are exposed to all treatments in different periods. Crossover experiments are generally used in psychology, the pharmaceutical industry, agriculture, and medicine.
Crossover designs are performed even with smaller sample sizes since the samples can act as their controls. These are better than simple randomized trials since patients are exposed to all the treatments.
Crossover designs are performed even with smaller sample sizes since the samples can act as their controls. These are better than simple randomized trials since patients are exposed to all the treatments.
2.8K
02:07Blind Procedures
10.6K
Ideally, the people who observe and record the children’s behavior are unaware of who was assigned to the experimental or control group, in order to control for experimenter bias. Experimenter bias refers to the possibility that a researcher’s expectations might skew the results of the study. Remember, conducting an experiment requires a lot of planning, and the people involved in the research project have a vested interest in supporting their hypotheses. If the observers knew which...
10.6K
02:01Group Design
8.9K
The most basic experimental design involves two groups: the experimental group and the control group. The two groups are designed to be the same except for one difference— experimental manipulation. The experimental group gets the experimental manipulation—that is, the treatment or variable being tested—and the control group does not. Since experimental manipulation is the only difference between the experimental and control groups, we can be sure that any differences between...
8.9K
01:13Randomized Experiments
6.9K
The randomization process involves assigning study participants randomly to experimental or control groups based on their probability of being equally assigned. Randomization is meant to eliminate selection bias and balance known and unknown confounding factors so that the control group is similar to the treatment group as much as possible. A computer program and a random number generator can be used to assign participants to groups in a way that minimizes bias.
Simple randomization
Simple...
Simple randomization
Simple...
6.9K
01:11Blinding
2.4K
Blinding is a commonly used method of not telling participants which treatment a subject is receiving. Blinding is a critical part of a randomized control trial or RCT. It reduces the bias that affects the results. In an RCT, blinding is used in the form of a placebo. A placebo effect occurs when untreated subjects falsely believe they have received the treatment and report improved symptoms. A placebo or a dummy treatment is administered to subjects to negate the bias caused by such an effect.
2.4K
01:21Friedman Two-way Analysis of Variance by Ranks
194
Friedman's Two-Way Analysis of Variance by Ranks is a nonparametric test designed to identify differences across multiple test attempts when traditional assumptions of normality and equal variances do not apply. Unlike conventional ANOVA, which requires normally distributed data with equal variances, Friedman's test is ideal for ordinal or non-normally distributed data, making it particularly useful for analyzing dependent samples, such as matched subjects over time or repeated measures...
194