The assessment of emergency physicians by a regulatory authority

To determine whether it is possible to develop a feasible, valid, and reliable multisource feedback program (360 degree evaluation) for emergency physicians.

Surveys with 16, 20, 30, and 31 items were developed to assess emergency physicians by 25 patients, eight coworkers, eight medical colleagues, and self, respectively, using five-point scales along with an "unable to assess" category. Items addressed key competencies related to communication skills, professionalism, collegiality, and self-management.

Data from 187 physicians who identified themselves as emergency physicians were available. The mean number of respondents per physician was 21.6 (SD +/- 3.87) (93%) for patients, 7.6 (SD +/- 0.89) (96%) for coworkers, and 7.7 (SD +/- 0.61) (95%) for medical colleagues, suggesting it was a feasible tool. Only the patient survey had four items with "unable to assess" percentages > or = 15%. The factor analysis indicated there were two factors on the patient questionnaire (communication/professionalism and patient education), two on the coworker survey (communication/collegiality and professionalism), and four on the medical colleague questionnaire (clinical performance, professionalism, self-management, and record management) that accounted for 80.0%, 62.5%, and 71.9% of the variance on the surveys, respectively. The factors were consistent with the intent of the instruments, providing empirical evidence of validity for the instruments. Reliability was established for the instruments (Cronbach's alpha > 0.94) and for each physician (generalizability coefficients were 0.68 for patients, 0.85 for coworkers, and 0.84 for medical colleagues).

The psychometric examination of the data suggests that the instruments developed to assess emergency physicians were feasible and provide evidence for validity and reliability.