Simulated patient judgements of medical student performance in OSCEs: A realist evaluation

Simulated patients (SPs) are widely used in Objective Structured Clinical Examinations (OSCEs) and increasingly contribute to summative scoring in high-stakes clinical assessments. While their judgments inform pass-fail decisions, concerns remain about variability and potential bias. There is limited theory-driven evidence explaining how SP involvement contributes to assessment decisions, and under what conditions this supports fair and credible assessment. This study aimed to develop and refine a programme theory explaining how SP involvement contributes to decision-making in high-stakes OSCEs.

We conducted a realist evaluation informed by an initial programme theory developed through a rapid realist review. Semi-structured interviews were undertaken with 19 SPs in a sequential OSCE at a UK medical school. Interviews explored how SPs approached scoring decisions and how features of the assessment environment shaped their judgments. Data were analysed iteratively using abductive and retroductive reasoning to identify context-mechanism-outcome configurations (CMOCs) and refine the programme theory.

Eighteen interrelated CMOCs were constructed across nine domains describing how SP judgments were shaped through interactions between assessment-related factors (including assessment tools, assessment focus, logistical environment, and the sequential format) and SP-related factors (including background and experience, training and benchmarking, role positioning, fatigue, and interaction with examiner). When SPs felt prepared, role clarity was established, and assessment tools and conditions were supportive, they reported making judgments experienced as fair and authentic. In contrast, high cognitive load, role ambiguity, fatigue and insufficient preparation activated mechanisms that increased perceived variability and potential for bias.

SP judgments in high-stakes OSCEs are complex and context-dependent. This programme theory provides an explanatory account of how SP judgments are produced and identifies conditions that support or undermine assessment. The findings offer practical implications for assessment design, SP training and faculty-SP collaboration to strengthen the use of SPs as assessors in high-stakes clinical examinations.