Related Experiment Video
Updated: Mar 14, 2026
15:00
A Tablet-Based Curriculum-Based Measurement Protocol for Kindergarten Writing
Published on: February 7, 2025
1.2K
Setting standards in knowledge assessments: Comparing Ebel and Cohen via Rasch
Matt Homer1, Jonathan C Darling1
1a Leeds Institute of Medical Education , School of Medicine, University of Leeds , Leeds , UK.
Medical Teacher
|September 22, 2016
Summary
The Ebel method consistently sets knowledge test standards across cohorts, unlike the Cohen method. A combined approach using multiple data sources is recommended for accurate standard setting.
Area of Science:
- Educational Measurement
- Psychometrics
Background:
- Test-centered standard-setting methods like Ebel are problematic due to inconsistent expert difficulty predictions.
- The Cohen method offers an alternative by benchmarking test difficulty against top candidate performance.
Purpose of the Study:
- To compare the consistency of the Ebel and Cohen methods for standard setting in knowledge tests across adjacent cohorts.
- To evaluate the impact of Rasch analysis and common anchor items on equating test standards.
Main Methods:
- Utilized common anchor items to link knowledge tests administered to two adjacent cohorts.
- Employed Rasch analysis to place all items and candidates on a common scale for comparison.
- Applied both the Ebel and Cohen methods for standard setting.
Main Results:
- Both Ebel and Cohen methods indicated similar test standards, but cohort abilities differed.
- The Ebel method demonstrated consistency across cohorts, while the Cohen method showed less consistency.
- Rasch equating presented challenges, including model misfit and anchor item difficulty changes.
Conclusions:
- Advocate for a pluralistic and pragmatic approach to educational standard setting.
- Recommend using multiple information sources to determine appropriate test standards.
Related Concept Videos
01:21Friedman Two-way Analysis of Variance by Ranks
535
Friedman's Two-Way Analysis of Variance by Ranks is a nonparametric test designed to identify differences across multiple test attempts when traditional assumptions of normality and equal variances do not apply. Unlike conventional ANOVA, which requires normally distributed data with equal variances, Friedman's test is ideal for ordinal or non-normally distributed data, making it particularly useful for analyzing dependent samples, such as matched subjects over time or repeated measures...
535
01:09Wilcoxon Signed-Ranks Test for Matched Pairs
568
The Wilcoxon signed-rank test for matched pairs evaluates the null hypothesis by combining the ranks of differences with their signs. It essentially tests whether the median of the differences in a population of matched pairs is zero. Since the test incorporates more information than the sign test, it generally yields more trustable conclusions. This test also does not require the data to follow a normal distribution, but two conditions must be met for it to be applicable: (1) the data must...
568
01:17Cochran's Q Test
1.1K
Cochran's Q Test is a nonparametric statistical test used to determine if there are potential differences in the outcomes of three or more related groups on a binary (yes/no) or dichotomous outcome. It is essentially an extension of the McNemar Test, which is limited to two related samples - Cochran's Q test can handle three or more related samples, making it more versatile in scenarios where subjects are measured under multiple conditions. The test statistic follows a Chi-Square...
1.1K
01:20Spearman's Rank Correlation Test
1.6K
Spearman's rank correlation test, also known as Spearman's rho, is a nonparametric method for assessing the strength and direction of association between two variables. This test is particularly valuable when the data distribution is unknown or when the assumption of normality does not hold. Named after the English psychologist and statistician Dr. Charles Edward Spearman, it serves as the nonparametric counterpart to Pearson's correlation coefficient.
Spearman's test calculates correlation by...
Spearman's test calculates correlation by...
1.6K
01:14Wilcoxon Signed-Ranks Test for Median of Single Population
494
The Wilcoxon signed-rank test for the median of a single population is a nonparametric test used to evaluate whether the median of a population differs from a specified value. Unlike parametric tests, it does not require data to follow a normal distribution, making it suitable for non-normal or small samples. The test begins by calculating the difference (d) between each observation and the hypothesized median. The absolute values of these differences are ranked in ascending order, with ties...
494
01:20Comparing the Survival Analysis of Two or More Groups
688
Survival analysis is a cornerstone of medical research, used to evaluate the time until an event of interest occurs, such as death, disease recurrence, or recovery. Unlike standard statistical methods, survival analysis is particularly adept at handling censored data—instances where the event has not occurred for some participants by the end of the study or remains unobserved. To address these unique challenges, specialized techniques like the Kaplan-Meier estimator, log-rank test, and...
688