Promoting Fairness in Medical School Assessment: Application of Differential Item Functioning Analysis

Achieving equity in assessment remains a critical goal for medical education. Differential Item Functioning (DIF) occurs when people from different groups, who have the same overall ability, respond differently to a test item, suggesting the item may be biased or unfair toward one group. The goals of this study were to determine the incidence of DIF on a high-stakes final exam in the pre-clerkship curriculum, explore potential causes of DIF, and assess whether faculty could revise items to eliminate DIF without compromising their psychometric properties. Baseline DIF was calculated for 115 test items used in 2022 and 2023. Using a logistic regression approach, 39 out of the 115 items were found to have DIF that was statistically significant, although in some cases the magnitude of the DIF was small. In 15 of the 39 items, the change in R² was greater than .035, indicating that the item had "moderate" or higher DIF. These 15 questions were edited by faculty, and tested using another cohort of students. Faculty were able to identify plausible hypotheses for DIF in 5 of the 15 edited items and proposed structural or semantic improvements for the others. Retesting with the 2024 cohort showed that DIF was eliminated in 13 of the 15 revised items, with no substantial changes in item difficulty or discrimination. However, the appearance of DIF in 19 of 98 reused unedited items in 2024 introduced a confounding variable, which is discussed further. Student focus groups reviewed a subset of items and supported faculty interpretations of DIF. They also reached broad consensus on strategies to improve item quality in terms of structure and language, towards reducing extraneous cognitive load. Our findings offer a practical classroom application that others may adopt and suggest a promising pathway for advancing equity in medical education assessment.