Time to normalise protected characteristics in written assessments: A pilot study

With increasing endeavours to incorporate teaching material on healthcare for underrepresented groups into medical school curricula, there remains a lack of research exploring the integration of this into assessments. With age and gender as the only demographic information routinely provided in single best answer (SBA) questions, this does not represent the diversity of patients encountered by doctors in clinical practice. This pilot study assessed how representation of patient demographic characteristics influenced medical students answering SBA questions.

200 medical students in their clinical years completed 15 SBA questions in an online simulated exam. Participants were randomised to control and test groups. The control group denoted age and gender, with test groups including neutral stems denoting age only; stems denoting LGBTQ+ identities; stems denoting ethnicity; and a mix of stems. Post-exam Likert scale questions explored their experience of the simulated exam.

Linear regression modelling demonstrated overall statistically nonsignificant differences between groups, indicating that assignment to control or test group does not explain differences in exam performance. Using a mix of question stems produced statistically significant differences, with participants scoring worse on question stems including LGBT+ identities and ethnicity.

As a pilot study these results suggest diversification of SBA questions does not inherently disadvantage students. A multimodal approach is needed to integrate diversity into medical school curricula and assessments. SBA questions can form part of this by mirroring diverse patient groups encountered in real-life clinical practice and positioning them as active agents seeking and accessing health care.