Incorporating simulation-based objective structured clinical examination into the Israeli National Board Examination...

We describe the unique process whereby simulation-based, objective structured clinical evaluation (OSCE) has been incorporated into the Israeli board examination in anesthesiology. Development of the examination included three steps: a) definition of clinical conditions that residents are required to handle competently, b) definition of tasks pertaining to each of the conditions, and c) incorporation of the tasks into hands-on simulation-based examination stations in the OSCE format, including 1) trauma management, 2) resuscitation, 3) crisis management in the operating room, 4) regional anesthesia, and 5) mechanical ventilation. Members of the Israeli Board of Anesthesiology Examination Committee assisted by experts from the Israel Center for Medical Simulation and from Israel's National Institute for Testing and Evaluation were involved in this process and in the development of the assessment tools, orientation of examinees, and preparation of examiners. The examination has been administered 4 times in the past 2 yr to 104 examinees and has gradually progressed from being a minor part of the oral board examination to a prerequisite component of this test. The pass rate ranged from 70% in resuscitation to 91% in regional anesthesia. The mean inter-rater correlations for all the checklist items, for the score based on the critical checklist items only, and for the general rating were 0.89, 0.86, and 0.76, respectively. The overall Kappa coefficients (the inter-rater agreement coefficient) for the total score and the critical checklist items were 0.71 and 0.76, respectively. The correlation between the total score and the general score was 0.76. According to a subjective feedback questionnaire, most (70%-90%) participants found the difficulty level of the examination stations reasonable to very easy and prefer this method of examination to a conventional oral examination. The incorporation of OSCE-driven modalities in the certification of anesthesiologists in Israel is a continuing process of evaluation and assessment.