Time to normalise protected characteristics in written assessments: A pilot study

Area of Science:

• Medical Education
• Health Equity
• Assessment in Medical Training

Background:

• Medical school curricula are increasingly incorporating healthcare for underrepresented groups.
• Current assessments, like single best answer (SBA) questions, often lack diverse patient demographic representation beyond age and gender.
• There is a gap in research on integrating diverse patient demographics into medical school assessments.

Purpose of the Study:

• To assess the influence of representing diverse patient demographic characteristics in single best answer (SBA) questions on medical students' performance.
• To explore the feasibility of integrating broader patient diversity into medical school assessments.

Main Methods:

• A pilot study involving 200 clinical-year medical students completing 15 online SBA questions.
• Students were randomized into control (age and gender) and test groups (age only, LGBTQ+ identities, ethnicity, or a mix).
• Post-exam surveys using Likert scales evaluated student experiences.

Main Results:

• Overall, no statistically significant differences in exam performance were found between control and test groups.
• However, using a mix of question stems, particularly those including LGBT+ identities and ethnicity, resulted in statistically significant lower scores.
• Linear regression indicated that group assignment did not explain performance differences.

Conclusions:

• Diversifying single best answer (SBA) questions with varied patient demographics does not appear to inherently disadvantage students.
• A multimodal strategy is necessary for integrating diversity into medical education and assessments.
• Single best answer (SBA) questions can be adapted to reflect diverse patient populations, enhancing clinical relevance.