Related Experiment Video
Updated: Aug 6, 2026
Emergency Undocking in Robotic Surgery: A Simulation Curriculum
Published on: May 20, 2018
Surgical Professionalism: Development and Educational Suitability Assessment of a Combined Situational Judgment Test
Daniel Ardian Soeselo1,2, Rennie Yolanda3, Patricia Winona3
1Department of Surgery and Medical Education Unit, School of Medicine and Health Sciences, Atma Jaya Catholic University of Indonesia, Jakarta, Indonesia.
Introduction:
Assessing professionalism in a valid, objective, and cost-effective manner remains a persistent challenge in clinical education. In Indonesia, this issue is particularly relevant in surgical training, where residents must integrate technical competence with professional behaviour in high-stakes, resource-limited settings. This study aimed to develop and evaluate the educational suitability of a Situational Judgement Test (SJT) combined with guided written reflection to assess and promote professionalism among general surgery residents.
Methods:
A mixed-methods sequential design was used. An SJT was developed through blueprinting, item writing, and expert validation, resulting in 26 scenarios across six professionalism domains. Residents (n = 64 at baseline; n = 44 completers) completed a proctored pre-test SJT, participated in an 8-week guided reflection program (four submissions), and completed an asynchronous post-test SJT. Educational suitability evidence was examined through content validity (CVI) and internal consistency (Kendall's W). No construct or criterion validity testing was performed. Reflections were scored using Kember's four-level reflective framework by three independent assessors with Fleiss' Kappa for inter-rater agreement. Changes in SJT and reflection depth were analysed using paired tests and correlation analysis. Data were analysed using SPSS version 23.
Results:
The mean CVI across items was 0.97 (range 0.90-1.00). The mean SJT score increased significantly from 386.59 ± 19.31 to 420.45 ± 12.56 among completers, with a mean difference of 33.86 ± 16.03 (p < 0.001). Reliability improved from Kendall's W = 0.38 to 0.52. Sensitivity analysis comparing baseline pre-test scores between completers (386.6 ± 19.3) and dropouts (391.0 ± 15.5) showed no significant difference (Mann-Whitney U, p = 0.258), indicating limited attrition bias. Reflective depth progressed from median level 1 to 4 across four submissions (median Δ = +3, p < 0.001), with substantial inter-rater agreement (κ = 0.68). There was no significant correlation between Δ reflection and Δ SJT (r = 0.059, p = 0.703).
Conclusions:
Integrating SJT with guided reflection demonstrated strong content validity, internal consistency, and educational benefit, supporting its suitability for formative educational use in surgical professionalism training.
