Multi-Modal Deep Learning for Assessing Surgeon Technical Skill.

Area of Science:

• Medical Education
• Artificial Intelligence in Medicine
• Surgical Skill Assessment

Background:

• Objective assessment of technical skill is crucial but resource-intensive in surgical education.
• Current methods for surgical skill assessment often rely on subjective human evaluation.
• Developing automated, objective tools for skill assessment is a growing need.

Purpose of the Study:

• To introduce a novel dataset for surgical knot-tying tasks.
• To develop and evaluate a multi-modal deep learning model for automated surgical skill assessment.
• To compare the performance of deep learning models against expert human raters.

Main Methods:

• Collected video, kinematic, and image data from 72 surgical trainees and faculty performing a knot-tying task.
• Developed three deep learning models: ResNet-based image, ResNet-LSTM kinematic, and a multi-modal fusion model.
• Evaluated model performance using the Objective Structured Assessment of Technical Skill (OSATS) Global Rating Scale (GRS), Mean Squared Error (MSE), and Intraclass Correlation Coefficient (ICC).

Main Results:

• All developed deep learning models demonstrated performance comparable to expert human raters on most GRS domains.
• The multi-modal deep learning model achieved the best overall performance.
• The models showed state-of-the-art performance in surgical skill assessment.

Conclusions:

• Multi-modal deep learning models show significant potential to accurately replicate human expert raters in assessing surgical skills.
• This research represents a key advancement towards automated surgical skill assessment.
• Automated assessment can reduce the burden on surgical training faculty and institutions.