Toward a navigation framework for fetoscopy.

Area of Science:

• Medical Imaging
• Computer Vision
• Surgical Technology

Background:

• Twin-to-twin transfusion syndrome (TTTS) requires precise surgical intervention.
• Fetoscopic laser photocoagulation is the standard treatment for TTTS.
• Enhanced visualization of the placenta is crucial for effective fetal surgery.

Purpose of the Study:

• To develop a learning-based framework for expanding the field-of-view during fetoscopic procedures.
• To create robust placental mosaics from intra-operative video frames for improved surgical exploration.
• To enable occlusion recovery and frame relocalization without relying on anatomical landmarks.

Main Methods:

• A novel framework utilizing learning-based features and keypoints for image registration.
• Transformer-based image-feature matching for robust mosaicking.
• Occlusion recovery and camera tracking loss mitigation using computed features.

Main Results:

• The framework achieved superior performance ([Formula: see text]) on 7 out of 10 in-vivo TTTS videos compared to state-of-the-art methods.
• Demonstrated a high success rate ([Formula: see text]) in occlusion recovery.
• Successfully computed placental panoramas and recovered from tracking loss.

Conclusions:

• The proposed learning-based framework offers robust placental mosaicking and occlusion recovery for TTTS surgery.
• It provides significant field-of-view expansion, crucial for surgical navigation.
• This technology has the potential to advance surgical navigation systems for fetal interventions.