Towards an Action Recognition Framework for Endovascular Surgery.

Area of Science:

• Medical Imaging
• Artificial Intelligence in Surgery
• Endovascular Interventions

Background:

• Objective assessment of instrument maneuvers in endovascular surgery is crucial for skill evaluation and technological advancement.
• Current methods primarily rely on laboratory settings, neglecting valuable clinical fluoroscopy data.
• There is a need for automated analysis of clinical fluoroscopy to understand surgical actions.

Purpose of the Study:

• To develop and validate a learning-based framework for motion activity recognition in clinical fluoroscopy sequences.
• To pioneer automated recognition of instrument maneuvers from real-world surgical data.
• To establish a foundation for image-based, automated assessment of endovascular procedures.

Main Methods:

• A novel deep learning framework was designed, comprising two networks: one for instrument segmentation and another for action recognition.
• The framework was applied to analyze fluoroscopy data from endovascular procedures.
• Feasibility was demonstrated using ex vivo datasets.

Main Results:

• The proposed framework successfully demonstrated the feasibility of automatically recognizing instrument maneuvers in fluoroscopy sequences.
• The system achieved accurate motion activity recognition from clinical data.
• Preliminary results show the potential for real-time analysis of surgical actions.

Conclusions:

• The developed framework enables image-based and automated assessment of endovascular tasks using fluoroscopy data.
• This technology facilitates the development of robotic control systems for minimally invasive surgery.
• It also supports enhanced surgical education and intelligent clinical documentation.