Robust tracking of deformable anatomical structures with severe occlusions using deformable geometrical primitives

Area of Science:

• Robotics and Computer Vision
• Medical Imaging and Computer-Aided Surgery

Background:

• Cognitive control architectures in surgical robotics aim to enhance patient safety and outcomes by enabling autonomy and reducing surgeon cognitive load.
• Workspace perception, including accurate anatomical structure detection and tracking, is crucial for automated decision-making in surgery.
• Minimally invasive surgery presents challenges such as limited visibility, occlusions, anatomical deformations, and camera movements that hinder robust detection and tracking.

Purpose of the Study:

• To develop a robust methodology for real-time detection and tracking of anatomical structures.
• To enable automatic control of robotic systems and enhance augmented reality applications in surgery.
• To validate the methodology in the context of fetoscopic repair of Open Spina Bifida.

Main Methods:

• A two-step methodology involving Convolutional Neural Network (CNN) for contour point selection.
• Reconstruction of anatomical shapes using deformable geometric primitives.
• Real-time detection and tracking for integration into robotic control and augmented reality.

Main Results:

• Validation through synthetic and real-scenario experiments, including extreme conditions.
• Demonstrated safety margins under nominal surgical conditions.
• Achieved accuracy, robustness, and computational efficiency in detecting and tracking anatomical structures.

Conclusions:

• The presented methodology offers robust anatomical structure detection despite camera movements, occlusions, and deformations.
• Applicable beyond the Open Spina Bifida case study to anatomies with contours approximable by geometric primitives.
• Provides effective inputs for cognitive robotic control and augmented reality systems requiring precise tracking of sensitive anatomies.