Automatic gesture recognition and evaluation in peg transfer tasks of laparoscopic surgery training

Area of Science:

• Medical Education
• Artificial Intelligence in Surgery
• Surgical Skill Assessment

Background:

• Laparoscopic surgery training requires objective assessment methods.
• Manual video annotation is time-consuming for instructors.
• Developing automated tools can alleviate annotation burdens and standardize evaluation.

Purpose of the Study:

• To develop an automatic surgical gesture recognition model for laparoscopic training.
• To create a gesture vocabulary for describing surgical procedures.
• To enable objective, AI-driven evaluation of surgical skills.

Main Methods:

• A gesture vocabulary was defined based on the Chinese Laparoscopic Skills Testing and Assessment (CLSTA) tool for the peg transfer task.
• A 3D ResNet-18 convolutional neural network (CNN) was employed for initial gesture recognition.
• An LSTM neural network was integrated to refine the sequence output, using a dataset of 80 videos.

Main Results:

• The 3D ResNet-18 model achieved 83.8% accuracy and an 84% F1 score.
• The addition of an LSTM network improved performance to 85.84% accuracy and an 85% F1 score.
• Analysis revealed performance variations in specific gestures (G1, G3, G6), highlighting areas for targeted trainee improvement.

Conclusions:

• An AI-powered model for automatic surgical gesture recognition in laparoscopic peg transfer was successfully developed.
• A defined gesture vocabulary enables sequential description and AI-based analysis of surgical training operations.
• This technology offers a pathway for objective, automated surgical skill evaluation in clinical settings using the CLSTA framework.