Investigating the Impact of Item Writing Flaws on Student Performance in Physiology Assessments at Two Institutions

Graduate and professional students are frequently evaluated through multiple-choice question (MCQ) examinations; however, MCQs may present challenges for test-takers when they contain item writing flaws (IWFs). IWFs potentially compromise the fairness and validity of the assessment. This study investigates how IWFs may impact test outcomes at two institutions: the University of Central Florida (UCF) and Boston University (BU). A total of 31 BU dental and graduate students and 54 UCF medical students completed a 20-question physiology assessment and demographic survey. An evaluation instrument based on the National Board of Medical Examiners (NBME) item writing guide was employed to identify the presence of IWFs in each question. Quantitative analysis examined demographics and their possible effects on item analysis factors (item difficulty and item discrimination). Test performance did not significantly differ between demographic groups, and item difficulty was not significantly related to the number of flaws at either institution. However, a moderately positive correlation between flaws and item discrimination was found at BU, suggesting that high-performing students may better handle IWFs. BU student performance also showed that items that contain unclear stems and lead-ins had more positive point biserial correlations, indicating questions with this IWF may disproportionately challenge weaker students. Assessments play an important role in decision-making processes and students' career trajectories. Ensuring assessments are equitable may contribute to diversifying the healthcare workforce needed to match the increasingly diverse U.S. population. These findings highlight the importance of question design in assessments to ensure equitable student outcomes.

The online version contains supplementary material available at 10.1007/s40670-026-02680-0.