Objective assessment of surgical performance in otolaryngology residency

A surgeon's technical competence is a key predictor of surgical outcomes, yet residents' performance is often assessed by direct observation without structured, predefined criteria, which can be biased and unreliable. This study aimed to implement structured assessment tools across multiple procedures in an ENT residency, assess their feasibility, and evaluate their impact on resident performance.

Two procedures per residency year were selected. Seven validated tools were adopted from the literature and an eighth was newly developed; each comprised a Task-Specific Checklist (TSC) and a Global Rating Scale (GRS). Residents were assessed over two rounds, each by three physicians plus a self-evaluation. The primary outcomes were implementation of the eight grids and the feasibility of multi-evaluator, multi-procedure assessment in routine practice. The secondary outcome was impact on performance measured as the change in TSC and GRS scores between rounds and the faculty-self agreement.

All eight grids were implemented and digitized; assessment was feasible for six of eight procedures, as endoscopic sinus surgery and pediatric bronchoscopy yielded too few eligible cases. No significant between-round differences were found for individual procedures, though non-significant improvement trends were observed. Pooled self-assessments improved significantly over time. Self- and faculty assessments did not differ significantly, though limited statistical power precludes firm conclusions. Junior residents improved most, whereas senior residents showed a ceiling effect.

This pilot study demonstrates the feasibility of structured, multi-evaluator surgical assessment in an ENT residency. Using multiple evaluators limits rater bias and will enable inter-rater reliability testing, not feasible at this stage. These preliminary results support a larger, multi-center study incorporating reliability and validity testing before broader implementation.