STANet: A Surgical Gesture Recognition Method Based on Spatiotemporal Fusion.

Area of Science:

• Robotics
• Computer Vision
• Artificial Intelligence

Background:

• Surgical gesture recognition is crucial for evaluating surgical quality and providing intelligent assistance in robotic surgery.
• Current deep learning models often struggle to effectively fuse spatial and temporal features, limiting their ability to capture long-term dependencies in action sequences.

Purpose of the Study:

• To propose a novel spatiotemporal adaptive network (STANet) for enhanced surgical gesture recognition.
• To effectively fuse spatial and temporal features for improved modeling of surgical action sequences.

Main Methods:

• Designed separate temporal and spatial modules for feature extraction.
• Fused extracted features and refined them using temporal adaptive convolution for integrated temporal modeling.
• Integrated the combined modules into a backbone network to create STANet.

Main Results:

• Validated STANet on JIGSAWS and RARP-45 surgical gesture datasets.
• Achieved superior performance compared to existing benchmark models.
• Demonstrated effective integration of long-term and short-term characteristics of surgical gestures.

Conclusions:

• STANet offers an efficient approach to modeling surgical action sequences by effectively fusing spatiotemporal features.
• The proposed method shows significant potential for applications in surgical robots, visual feedback systems, and computer-assisted surgery.