Skill-level classification and performance evaluation for endoscopic sleeve gastroplasty.

Area of Science:

• Medical simulation
• Surgical training
• Machine learning in healthcare

Background:

• Quantitative grading metrics were previously developed for simulated endoscopic sleeve gastroplasty (ESG) to differentiate experts from novices.
• This study expands on prior work by incorporating synthetic data generation and advanced machine learning techniques for skill level analysis.

Purpose of the Study:

• To develop and validate machine learning models for classifying endoscopists as experts or novices based on simulated ESG performance.
• To identify critical sub-tasks and optimize grading metrics for improved skill differentiation.

Main Methods:

• Synthetic Minority Over-sampling Technique (SMOTE) was used to augment and balance the dataset of simulated ESG procedures.
• Six machine learning classifiers (SVM, AdaBoost, KNN, KFDA, random forest, decision tree) were employed for expert-novice classification.
• An optimization model was developed to assign weights to tasks and maximize the separation between expert and novice performance scores.

Main Results:

• Classifiers achieved high accuracy, with training accuracies ranging from 0.94 to 1.00 and testing accuracy reaching 1.00 for SVM and AdaBoost.
• The optimization model significantly increased the separation between expert and novice scores, from an initial distance of 2 to 53.72.
• Feature reduction combined with algorithms like SVM and KNN demonstrated effectiveness in classifying endoscopists.

Conclusions:

• Machine learning, particularly with feature reduction and algorithms like SVM and KNN, can effectively classify endoscopists' skill levels in simulated ESG.
• Non-linear constraint optimization aids in separating expert and novice performance clusters and identifying key procedural tasks.