Using Intraoperative Recordings to Evaluate Surgical Technique and Performance in Mastoidectomy

Area of Science:

• Surgical Education
• Otolaryngology
• Medical Simulation

Background:

• Otolaryngology residency programs lack objective methods for surgical skill assessment, often relying on subjective evaluations.
• There is a need for reliable tools to evaluate surgeon proficiency during procedures like mastoidectomy.

Purpose of the Study:

• To determine if mastoidectomy techniques can indicate surgeon level.
• To assess if surgeon level can be determined by analyzing drill head or suction movements alone.

Main Methods:

• Prospective observational study of 24 intraoperative mastoidectomy recordings (January 2015-December 2019).
• Three independent observers performed blinded evaluations on original and animated videos (showing only drill/suction path).
• Assessed metrics included drill stroke count, drilling efficiency, stroke pattern, and suction/irrigation use.

Main Results:

• Intraclass correlation showed excellent reliability for drill stroke count, suction, irrigation, and estimated surgeon level from original videos.
• Animated video analysis demonstrated excellent reliability for drill stroke count and drilling efficiency.
• Junior residents exhibited lower drill stroke counts compared to senior residents and attending surgeons.

Conclusions:

• Intraoperative mastoidectomy recordings are a feasible method for evaluating surgeon proficiency.
• Drill stroke count, drilling efficiency, and stroke pattern are reliable indicators of surgeon level.
• These recordings can supplement resident education and performance assessment.