Using Individual Clinical Evaluations to Assess Residents' Clinical Judgment; Feasibility and Residents' Perception

In surgical training, most assessment tools focus on advanced clinical decision-making or operative skill. Available tools often require significant investment of resources and time. A high stakes oral examination is also required to become board-certified in surgery. We developed Individual Clinical Evaluation (ICE) to evaluate intern-level clinical decision-making in a time- and cost-efficient manner, and to introduce the face-to-face evaluation setting.

Intern-level ICE consists of 3 clinical scenarios commonly encountered by surgical trainees. Each scenario was developed to be presented in a step-by-step manner to an intern by an attending physician or chief resident. The interns had 17 minutes to complete the face-to-face evaluation and 3 minutes to receive feedback on their performance. The feedback was transcribed and sent to the interns along with incorrect answers. Eighty percent correct was set as a minimum to pass each scenario and continue with the next one. Interns who failed were retested until they passed. Frequency of incorrect response was tracked by question/content area. After passing the 3 scenarios, interns completed a survey about their experience with ICE.

Beth Israel Deaconess Medical Center, an academic tertiary care facility located in Boston, Massachusetts.

All first-year surgery residents in our institution (n = 17) were invited to complete a survey.

All 2016-2017 surgical interns (17) completed the ICEs. A total of $171 (US) was spent conducting the ICEs, and an average of 17 minutes was used to complete each evaluation. In total, 5 different residents failed 1 scenario, with the most common mistake being: failing to stabilize respiration before starting management. After completing the 3 clinical scenarios, more than 90% of respondents agreed or strongly agreed that the evaluations were appropriately challenging for training level, and that the evaluations helped to identify personal strengths and weaknesses in skill and knowledge. The majority believed their knowledge improved as a result of the ICE and felt better prepared to manage these scenarios (88% and 76%, respectively).

The ICE is an inexpensive and time efficient way to introduce interns to board type examinations and assess their preparedness for perioperative patient care issues. Common errors were identified which were able to inform educational efforts. ICEs were well accepted by residents. Next steps include extension of the ICE to PGY2 and PGY3 residents.