Objective Structured Assessment of Technical Skills in Mastoidectomy Using 3D-Printed Temporal Bones

To evaluate the use of three-dimensional (3D) printed temporal bone models combined with final product analysis as an objective structured assessment of technical skills tool to assess mastoidectomy performance across levels of otolaryngology residency training.

In this prospective observational study conducted at a single academic otolaryngology residency program, 32 residents performed 64 mastoidectomies on 3D-printed temporal bone models over a three-year period. Three expert faculty independently and anonymously evaluated each performance using a validated 14-item final product analysis checklist. Statistical analyses included linear regression, ANOVA, Mann-Whitney U, Wilcoxon signed-rank tests, and intraclass correlation coefficient (ICC) for inter-rater reliability.

Median scores improved steadily with training level, from 35.0 in postgraduate year (PGY) 1 to 53.7 in PGY 5 out of 70. Linear regression demonstrated a significant positive association between both training level and duration in residency with performance. Senior residents (PGY 4-5, median score 54.0) significantly outperformed junior residents (PGY 1-3, median score 38.0; p = 0.012). Across all training levels, median scores improved from 43.2 on the first attempt to 52.5 on the third, though this difference was not statistically significant. Inter-rater reliability was excellent (ICC = 0.9210, p < 0.0001). A proficiency cutoff score of 45/70 was established using the contrasting groups method.

The consistency of 3D-printed temporal bones judged by final product analysis allows for fair, objective, and reliable assessment for formative and summative evaluations. By longitudinally tracking skill improvement and differentiating performance across training levels, it supports the transition to competency-based surgical education.

N/A.