Faculty evaluation by residents in an emergency medicine program: a new evaluation instrument

Evaluation of preceptors in training programs is essential; however, little research has been performed in the setting of the emergency department (ED). The goal of this pilot study was to determine the validity and reliability of a faculty evaluation instrument-the Emergency Rotation (ER) scale-developed specifically for use in emergency medicine (EM).

A prospective study comparing the ER scale with two alternative faculty evaluation instruments was completed in three of the five EDs affiliated with an EM teaching program, where emergency physicians are members of the clinical teaching faculty. The participants were 18 residents (postgraduate years 1, 2, and 3) who were completing four-week clinical rotations in EM. Residents at the end of the rotation recorded their evaluations of each emergency physician with whom they had clinical encounters on the following evaluation tools: the ER scale, a longer validated scale (Irby), and a global assessment scale (GAS). Domain scores were correlated with the previously validated scale and the GAS to determine validity using a multitrait-multimethod matrix. The reliability of the ER scale was measured using a Chronbach's alpha coefficient.

Forty-eight preceptor evaluations were completed on 29 individual preceptors. The rating of preceptors was high using the ER scale (median: 16 of 20; IQR: 13, 18), Irby (median: 300 of 378; IQR: 267, 321), or GAS (mean: 7.8 of 10; SD: 1.3). Domain scores for each tool were used in the multitrait-multimethod matrix and the correlations between a previously validated tool and the ER scale were found to be high (>0.70) in the various domains. The internal consistency of the ER scale was also high (r = 0.85).

The ER scale appears to be valid and reliable. It performs well when compared with previously psychometrically tested tools. It is a sensible, well-adapted tool for the teaching environment offered by EM.