3-D-Printed Models for Temporal Bone Training: A Validity Study

3-D printing offers convenient and low-cost mastoidectomy training; nonetheless, training benefits using 3-D-printed temporal bones remain largely unexplored. In this study, we have collected validity evidence for a low-cost, 3-D-printed temporal bone for mastoidectomy training and established a credible pass/fail score for performance on the model.

A prospective educational study gathering validity evidence using Messick's validity framework.

Seven Danish otorhinolaryngology training institutions.

Eighteen otorhinolaryngology residents (novices) and 11 experienced otosurgeons (experts).

Residents and experienced otosurgeons each performed two to three anatomical mastoidectomies on a low-cost, 3-D-printed temporal bone model produced in-house. After drilling, mastoidectomy performances were rated by three blinded experts using a 25-item modified Welling scale (WS).

Validity evidence using Messick's framework including reliability assessment applying both classical test theory and Generalizability theory.

Novices achieved a mean score of 13.9 points; experienced otosurgeons achieved 23.2 points. Using the contrasting groups method, we established a 21/25-point pass/fail level. The Generalizability coefficient was 0.91, and 75% of the score variance was attributable to participant performance, indicating a high level of assessment reliability. Subsequent D studies revealed that two raters rating one performance or one rater rating two performances were sufficiently reliable for high-stakes assessment.

Validity evidence supports using a low-cost, 3-D-printed model for mastoidectomy training. The model can be printed in-house using consumer-grade 3-D printers and serves as an additional training tool in the temporal bone curriculum. For competency-based training, we established a cut-off score of 21 of 25 WS points using the contrasting groups method.