Comparing performance and integrity of proctored physical, unproctored online, and proctored online MBBS exams

The rapid pivot to online assessment during COVID-19 raised concerns about the fairness and psychometric quality of high-stakes medical examinations. This study compared performance and integrity indices across three successive modalities of The University of the West Indies Final MBBS Medicine and Therapeutics written paper.

Institutional records for every candidate in 2019 (proctored physical, n = 508), 2020 (un-proctored online, n = 529) and 2021 (physically onsite proctored online, n = 361) were analyzed retrospectively (N = 1 398). Mean scores, score dispersion, categorical outcomes, Cronbach's alpha, and item-difficulty indices were contrasted with one-way ANOVA, χ² tests, and odds ratios.

Examination modality significantly influenced performance. The mean (± SD) scores were highest in the unproctored online exam (2020: 68.4 ± 6.5), followed by proctored physical exam (2019: 64.2 ± 7.2) and proctored online (2021: 60.5 ± 7.3). The difference in the mean score of the three modalities was highly significant (p < 0.0001). Pass rates peaked in 2020 (92%) versus 2019 (81%) and 2021 (70%). The odds of failing the examination in 2021 were more than five times higher than in 2020 (OR = 5.652). Reliability (Cronbach's α) highest in 2019 (0.813) and 2021 (0.770) but very low in 2020 (0.307). Odds of failing in 2021 were 5.65 times higher than 2020.

Examination modality strongly impacted on performance and reliability. Unproctored online exams led to grade inflation, reduced variability, and low reliability, while proctored formats-physical or virtual-reinstated assessment rigor, fairness, and psychometric integrity, emphasizing the need for robust invigilation in high-stakes medical assessments.