Assessment of operative competency in otolaryngology residency: Survey of US Program Directors

1) Assess current status of operative competency assessment and feedback among US Otolaryngology residency programs. 2) Evaluate correlations between assessment or feedback tools and remediation.

Survey of Program Directors.

The survey was sent to 100 Program Directors inquiring about program size, mode of feedback, assessment tools, surgical portfolios, remediation, and number of residents requiring remediation in past 5 years. Associations between assessment questions and remediation (yes or no) were statistically assessed with the [chi]2 goodness-of-fit test. For more complicated analyses, multiple logistic regression models were created to assess predictors of the binary outcome of radiation.

Among 72 respondents, subjective evaluations are used by 95.8% (69 of 72). Formative feedback after cases is used by 38.9% (28 of 72). Summative feedback at end of the rotation is done by 57.0% (41 of 72). Objective Structured Assessments of Technical Skills is used by 15.3% (11 of 72) of programs, most are large on the basis of number of residents or faculty or both. Operative portfolios are kept by 48.6% (35 of 72) of programs. The percentage of programs with mechanisms for remediation is 41.7% (30 of 72). Similarly, of the programs who responded to the remediation question, the percentage with at least one resident requiring remediation is 37.5%. Programs who report at least one resident remediation are statistically more likely to have a remediation mechanism in place (75% vs. 27.5%; P <.001). There is also a statistically significant association with formative feedback and having resident remediation (58.3% vs. 30%; P = .025).

Standardized surgical curricula and assessment tools are needed. Programs with more intensive evaluation or remediation mechanisms are statistically more likely to report residents requiring remediation. This may reflect their ability to better identify the struggling resident. Alternatively, having a resident requiring remediation may be the catalyst that initiates the need for more intensive evaluation tools and remediation mechanisms. The Accreditation Council for Graduate Medical Education's and society will demand that we produce competent surgeons. Therefore, all Otolaryngology training programs should be developing and implementing formal surgical assessment tools, methods to identify residents requiring remediation, and remediation mechanisms.