Using open surgery simulation kinematic data for tool and gesture recognition.

Area of Science:

• Surgical simulation and performance analysis.
• Machine learning applications in healthcare.
• Wearable sensor technology for skill assessment.

Background:

• Motion sensors offer a novel approach to objectively measure surgical performance.
• Assessing surgical skill traditionally relies on subjective evaluations.
• Automated analysis of surgical actions can enhance training and feedback.

Purpose of the Study:

• To identify surgical gestures and tools during open surgery suturing simulations using motion sensor data.
• To compare the effectiveness of different deep learning models for this task.
• To evaluate multi-task learning approaches for improved performance.

Main Methods:

• Twenty-five participants performed a suturing task using electromagnetic motion sensors.
• Compared performance of Gated Recurrent Unit (GRU), Long Short-Term Memory (LSTM), and MS-TCN++ networks.
• Adapted MS-TCN++ for motion sensor data and extended architectures for multi-task learning.

Main Results:

• MS-TCN++ achieved the highest accuracy and F1-Macro scores for gesture recognition.
• MS-TCN++ demonstrated superior performance in right-hand tool usage recognition.
• Multi-task GRU network excelled in left-hand tool usage recognition, with a smaller model size.

Conclusions:

• Motion sensor data can be effectively used for automatic identification of surgical gestures and tools in simulations.
• MS-TCN++ and multi-task GRU show promise for detailed surgical performance metrics and workflow analysis.
• The choice of model (MS-TCN++ vs. multi-task GRU) depends on the specific recognition task (gesture, right-hand tool, or left-hand tool).