Vision-based online recognition of surgical activities

Area of Science:

• Medical technology
• Surgical robotics
• Computer vision

Background:

• Surgical processes generate vast amounts of data, posing challenges for current vision-based recognition methods.
• Inefficiency in vision-based systems stems from the large volume of information captured during procedures.
• Existing methods struggle with extended recognition times and information overflow.

Purpose of the Study:

• To develop an efficient surgical gesture recognition system using temperature-based recognition.
• To overcome the limitations of current vision-based surgical activity monitoring.
• To enable real-time recognition of surgical activities through a novel approach.

Main Methods:

• An infrared thermal camera was integrated with a hierarchical temporal memory system.
• Hand temperatures were captured and analyzed to differentiate between warm hands and the colder environment.
• Gesture extraction and recognition were performed based on thermal differences.

Main Results:

• The system demonstrated 96% online recognition accuracy during simulated functional endoscopic sinus surgery.
• High precision and recall rates of approximately 60% were achieved.
• The system effectively recognized surgical activities based on thermal data.

Conclusions:

• Shifting from visible light to infrared spectrum overcomes information overflow and extended recognition times.
• Temperature-based recognition offers a viable alternative for efficient surgical gesture recognition.
• This system enables real-time monitoring for surgical assistance and workflow management.