Depth over RGB: automatic evaluation of open surgery skills using depth camera.

Area of Science:

• Medical Technology
• Surgical Simulation
• Computer Vision

Background:

• Automatic evaluation of open surgery skills commonly uses RGB cameras.
• Depth cameras present potential advantages including lighting robustness, privacy, and data efficiency.

Purpose of the Study:

• To evaluate depth cameras as an alternative to RGB cameras for automatic open surgery skill assessment.
• To demonstrate comparable performance of depth cameras in object detection and action segmentation.

Main Methods:

• Utilized YOLOv8 for tool detection and UVAST/MSTCN++ for action segmentation on RGB and depth videos.
• Collected and annotated a dataset using Azure Kinect, focusing on hand and tool interactions in suturing simulators.
• Compared expert and novice surgeon performance in suturing tasks.

Main Results:

• Depth cameras achieved comparable results to RGB cameras for object detection and action segmentation.
• Analysis of 3D hand path length revealed significant differences between expert and novice surgeons.
• Investigated camera angle influence, highlighting 3D cameras' accuracy in representing hand movements.

Conclusions:

• Depth cameras provide a reliable and privacy-preserving method for surgical skill assessment.
• Findings support the use of depth cameras in surgical training and evaluation.
• This research lays groundwork for future advancements in automated surgical skill analysis.