Related Experiment Video
Updated: Jul 14, 2026
05:21
Computerized Adaptive Testing System of Functional Assessment of Stroke
Published on: January 7, 2019
Promoting Fairness in Medical School Assessment: Application of Differential Item Functioning Analysis
Paul Sturgis1, Conrad Yee2, Arielle Patterson2
1Department of Medical Education, University of Central Florida, Orlando, FL 32827 USA.
Medical Science Educator
|July 13, 2026
Summary
Medical education assessments can be made more equitable by identifying and revising biased test items. Faculty successfully edited items with Differential Item Functioning (DIF), improving fairness in medical education exams.
Area of Science:
- Medical Education
- Psychometrics
- Educational Assessment
Background:
- Achieving assessment equity is crucial in medical education.
- Differential Item Functioning (DIF) indicates potential bias in test items.
- Identifying and mitigating DIF is essential for fair evaluation.
Purpose of the Study:
- To determine the incidence of DIF in a high-stakes pre-clerkship medical exam.
- To explore potential causes of DIF.
- To assess faculty's ability to revise DIF items without compromising psychometric properties.
Main Methods:
- Calculated baseline DIF for 115 test items across two cohorts.
- Used logistic regression to identify statistically significant DIF.
- Faculty revised 15 items with moderate or higher DIF; retested with a new cohort.
Main Results:
- 39 out of 115 items showed statistically significant DIF.
- 13 of 15 revised items had DIF eliminated post-revision.
- No substantial changes in item difficulty or discrimination were observed after revisions.
Conclusions:
- Faculty can effectively revise test items to eliminate DIF.
- Item revision offers a practical approach to enhance assessment equity in medical education.
- Student feedback supports strategies to reduce extraneous cognitive load and improve item quality.
Related Concept Videos
01:21Friedman Two-way Analysis of Variance by Ranks
Friedman's Two-Way Analysis of Variance by Ranks is a nonparametric test designed to identify differences across multiple test attempts when traditional assumptions of normality and equal variances do not apply. Unlike conventional ANOVA, which requires normally distributed data with equal variances, Friedman's test is ideal for ordinal or non-normally distributed data, making it particularly useful for analyzing dependent samples, such as matched subjects over time or repeated measures from...
01:15One-Way ANOVA: Unequal Sample Sizes
One-way ANOVA can be performed on three or more samples of unequal sizes. However, calculations get complicated when sample sizes are not always the same. So, while performing ANOVA with unequal samples size, the following equation is used:
01:15One-Way ANOVA: Equal Sample Sizes
One-Way ANOVA can be performed on three or more samples with equal or unequal sample sizes. When one-way ANOVA is performed on two datasets with samples of equal sizes, it can be easily observed that the computed F statistic is highly sensitive to the sample mean.
Different sample means can result in different values for the variance estimate: variance between samples. This is because the variance between samples is calculated as the product of the sample size and the variance between the...
Different sample means can result in different values for the variance estimate: variance between samples. This is because the variance between samples is calculated as the product of the sample size and the variance between the...
00:57Behrens–Fisher Test
The Behrens-Fisher test is a statistical method designed to address the Behrens-Fisher problem, which arises when comparing the means of two normally distributed populations with unequal variances. Unlike the Student's t-test, which assumes equal variances, the Behrens-Fisher test allows for mean comparison without this restrictive assumption. This flexibility makes it particularly valuable in scenarios where two independent samples exhibit normality but lack variance homogeneity.
This test is...
This test is...
01:14Identifying Statistically Significant Differences: The F-Test
The F-test is used to compare two sample variances to each other or compare the sample variance to the population variance. It is used to decide whether an indeterminate error can explain the difference in their values. The underlying assumptions that allow the use of the F-test include the data set or sets are normally distributed, and the data sets are independent of each other. The test statistic F is calculated by dividing one variance by another. In other words, the square of one standard...
01:23Test for Homogeneity
The goodness–of–fit test can be used to decide whether a population fits a given distribution, but it will not suffice to decide whether two populations follow the same unknown distribution. A different test, called the test for homogeneity, can be used to conclude whether two populations have the same distribution. To calculate the test statistic for a test for homogeneity, follow the same procedure as with the test of independence. The hypotheses for the test for homogeneity can be stated as...