Automated performance assessment in simulated laparoscopic crural repair

Crural repair involves reconstructing the esophageal hiatus. Automated AI-based assessment of intracorporeal suturing could improve trainee performance through objective feedback. We evaluated an AI system designed to assess performance in simulated laparoscopic crural repair without manual annotations.

In this institutional review board-approved study, tool motion was tracked from 47 videos of 33 participants (15 novices, 18 experts) performing simulated crural repair. Each suture placement was segmented from needle grasp to final knot, and bilateral tool motion was analyzed. Kinematic features path length, root mean square (RMS) velocity, jerk, and bimanual dexterity were extracted. Noise was removed using a 24 Hz low-pass filter. Machine learning models (logistic regression, random forest, support vector classifier, XGBoost) were trained using tenfold cross validation. An ablation study identified the top-performing model, and group differences were evaluated with the Mann-Whitney U test.

Data from all participants were successfully analyzed. Logistic regression with min-max scaling achieved the best performance (74% accuracy, an F1 score of 0.76). Significant differences between novice and expert groups were found for bimanual dexterity (p = 0.01), right-hand tool RMS velocity (p = 0.02), left-hand tool RMS jerk (p = 0.001), and total path length (p < 0.001), supporting the relevance of selected features.

The AI system reliably assessed skill level without manual annotation. This approach shows promise for broader application across laparoscopic procedures.